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ABSTRACT 

Exercise  Prowling  Pegasus  was  held  during  May  19th  to  the  22nd,  2001,  at  DSTO  Edinburgh. 
The  aim  of  the  exercise  was  to  support  ARH  doctrine  and  TTP  development,  and  the 
development  of  joint  Land  Air  Systems  concepts.  A  human  factors  behavioural  observation 
approach  was  taken  in  order  to  assess  the  behavioural  indicators  of  situation  awareness  (SA) 
and  the  effect  of  SA  enhancing  tools  on  the  performance  of  military  participants.  Results  on 
the  levels  of  behaviour,  the  outcomes  of  wargames,  and  the  relations  of  the  two  are  discussed. 
Future  work  aims  to  clarify  the  relationship  between  behavioural  observations  and  wargame 
outcomes. 
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Behavioural  Situation  Awareness  Measures  and  the 
Use  of  Decision  Support  Tools  in  Exercise  Prowling 

Pegasus 


Executive  Summary 

Exercise  Prowling  Pegasus  was  held  during  the  19th  to  the  22nd  May,  2001,  at  the  Land 
Operations  Division  (LOD)  of  DSTO  Edinburgh.  The  aim  of  the  exercise  was  to  support 
Armd  Reconnaissance  Helicopter  doctrine  and  Tactics,  Techniques  and  Proceduress 
development,  and  the  development  of  joint  Land  Air  Systems  concepts.  A  human 
factors  behavioural  observation  approach  was  taken  in  order  to  assess  the  behavioural 
indicators  of  situation  awareness  (SA)  and  the  effect  of  SA  enhancing  tools  on  the 
performance  of  military  participants.  Participants  were  military  staff  from  a  variety  of 
locations  within  Australia,  as  well  as  military  staff  from  within  LOD  itself. 

Results  on  the  levels  of  behaviour,  the  outcomes  of  wargames,  and  the  relations  of  the 
two  are  discussed  in  terms  of  levels  of  behaviour,  correlations  with  other  behaviours 
and  a  basic  outcome  measure  (the  kill/loss  ratio  in  this  instance).  Trends  of  behaviour 
level  and  outcome  results  are  shown  graphically  and  discussed.  There  are  no 
significant  relationships  obvious  between  the  behaviour  and  outcome  measures  taken 
during  this  study,  although  the  trends  observed  tend  to  be  indicative  of  associations 
between  these  measures  which  may  be  elucidated  using  differently  structured 
experments.  Certain  expected  trends  were  observed,  as  in  the  increase  in  levels  of 
information  handling  behaviours  during  or  immediately  following  the  occurrence  of 
critical  events.  These  observations  also  show  that  the  rises  in  behaviour  occur  most 
prominently  in  the  critical  area  of  the  moment  (eg.  when  an  event  makes  the  airborne 
component  of  the  Land  Air  System  vital,  the  changes  to  behaviour  levels  are  most 
obvious  in  the  Aviation  Regimental  Headquarters).  Clear  preferences  for  the  use  of 
radios  as  communication  tools  are  still  observed  in  two  of  the  four  locations,  indicating 
a  preference  for  voice  contact  when  passing  information  to  other  staff.  The  introduced 
SA  tool  was  the  preferred  tool  for  briefings,  particularly  in  the  Brigade  Headquarters. 
These  results  can  be  expected  given  the  relatively  unfamiliar  nature  of  the  SA  tool, 
making  it  less  efficient  than  voice  communications  for  the  transmission  of  information 
at  this  stage. 

Future  work  stemming  from  the  current  project  aims  to  clarify  the  relationship 
between  behavioural  observations  and  wargame  outcomes,  in  addition  to  employing  a 
more  in-depth  approach  to  the  representation  of  relevant  behaviours.  This  approach  — 
known  as  the  "behaviour  systems"  methodology  —  structures  the  observed  and 
expected  behaviours  in  a  way  that  triggers  feedback  loops,  and  the  consequences  of 
changes  to  the  environment  can  be  pinpointed.  This  would  enable  remedial  steps  to  be 
taken  in  a  situation  where  introduced  technology  or  changes  in  processes  have  resulted 
in  disruption  of  the  performance  levels  of  subjects.  Clearly,  the  value  here  is  the  ability 
to  analyse  the  source  of  problems  and  to  take  steps  to  remedy  the  implementation  of 
tools  and  technologies  in  order  to  avoid  such  occurrences  in  the  future  and  ensure  that 
such  tools  enhance  the  functioning  of  the  military  as  a  team. 
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1.  Introduction 

As  part  of  the  ongoing  work  within  the  Land  Operations  Division  (LOD)1  on  enhancing 
Situation  Awareness  (SA),  data  collection  for  a  Human  Factors  assessment  of  shared  SA 
and  the  use  of  technologically  advanced  SA  tools  was  carried  out  during  Exercise 
Prowling  Pegasus  (conducted  at  DSTO  Edinburgh,  19th  to  23rd  March  2001).  This  report 
presents  the  results  of  the  Human  Factors  analysis  of  behavioural  indicators  of  SA  and 
examines  their  relationship  with  incidents  occurring  during  the  exercise,  the  achievement 
of  the  commander's  intent,  and  the  use  of  tools  during  the  missions. 

Recent  years  have  seen  the  continuation  of  work  conducted  in  the  area  of  enhanced  SA  in 
LOD:  three  key  foci  within  this  comprehensive  area  are  the  generation  of  knowledge  from 
information,  the  Information  Management  and  Dissemination  Architecture  (IMDA)  to 
support  SA  for  various  decision-makers,  and  a  methodology  for  developing  System  of 
Systems  (SoS)2  emergent  behaviour  and  measuring  the  impact  on  force  effectiveness 
(Seymour,  Grisogono,  Unewisse,  Johnson,  Krieg,  and  Haub,  2000).  In  the  midst  of  the 
development  of  such  tools,  and  their  introduction  to  the  Land  Force  as  potential  methods 
of  shortening  the  decision  cycle  (in  order  to  work  within  the  enemy's  OODA3  loop),  there 
are  additional  aspects  to  be  considered.  Evaluating  the  effects  of  these  tools  on  the 
performance  of  the  force(s),  and  the  usability  and  design  of  the  tools  themselves,  for 
example,  are  areas  where  the  Human  Factors  contribution  can  be  valuable. 

When  the  performance  of  tools  within  the  team  context — as  the  Military  can  be  termed  — 
refers  to  the  overall  performance  of  the  human-machine  system,  assessments  can  be 
conducted  on  the  behaviours  performed  by  the  team  members  in  order  to  carry  out 
particular  important  functions  (eg.  developing  and  maintaining  SA).  That  is,  behaviours 
displayed  by  team  members  that  are  designed  to  develop  and  maintain  SA  can  be 
examined,  and  compared  with  those  displayed  when  new  technologies  are  added  to  the 
system.  To  be  effective,  such  technologies  should  enhance  the  speed  at  which  SA  is 
developed  and  how  fluidly  it  is  maintained,  and/or  the  quality  of  the  SA  shared  by 
commanders  and  staff.  In  addition  these  tools  should  allow  commanders  and  staff  to 
produce  the  necessary  SA  within  a  shorter  time  frame  than  previously  observed. 

Recently,  the  focus  of  such  research  has  become  the  question  'what  tools,  functionalities 
and  combinations  of  functionalities  will  be  of  maximum  benefit  to  distributed  teams  or 
teams  of  teams  (ToT)?'  When  commanders  need  to  maintain  their  overall  understanding  of 
a  situation  and  their  battlespace  awareness,  and  they  are  working  with  combined  forces 
[which  may  include  land,  sea  and  air  troops],  what  are  the  requirements  for  effectively 


1  DSTO  Edinburgh,  South  Australia. 

2  Here,  the  SoS  is  one  involving  multiple  information  sources  -  such  as  Intelligence,  Surveillance 
and  Reconnaissance  (ISR)  -  and  tools  for  accessing,  processing  and  acting  on  information  (known  as 
C4  technologies:  Command,  Control,  Communications  and  Computers).  Refer  to  Seymour  et  al 
(2000)  for  further  detail. 

3  Observe,  Orient,  Decide,  Act. 
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maintaining  this  awareness?  In  this  case,  the  Human  Factors  researcher  might  ask  'what 
aspects  of  behaviour  are  essential  to  effective  SA  maintenance,  and  how  can  the 
functioning  of  these  be  enhanced  via  the  systematic  introduction  of  technology?'  In  order 
to  answer  these  questions,  it  is  necessary  to  delineate  the  types  of  behaviour  carried  out,  at 
what  levels  these  behaviours  occur,  how  they  may  relate  to  critical  incidents  during  the 
battle,  and  what  relationship  the  tools  provided  may  have  with  these  behaviours  (ie.  is 
there  an  interaction,  and  if  so,  is  it  effective). 

1.1  Behavioural  SA  indicators 

The  data  collection  instruments  used  for  the  exercise  consisted  of  a  checksheet  containing 
a  list  of  behaviours  relating  to  aspects  of  SA  development  and  maintenance,  and  a 
checksheet  consisting  of  a  log  of  tool  use  by  the  participants.  These  are  described  more 
fully  in  section  2.3. 

The  behaviour  list  was  initially  developed  from  one  constructed  by  Muniz,  Stout,  Bowers, 
and  Salas  (1998)  for  use  in  the  SALIANT  methodology  for  measuring  team  SA.  The 
behaviours  (shown  in  Table  1)  are  theoretically  based  indicators  of  SA,  and  thus  a  list  of 
physically  observable  behaviours  needed  to  be  extrapolated  from  these.  This  was  carried 
out  using  experience  from  previous  observations  to  interpret  and  expand  on  the  list  put 
forward  by  Muniz  et  al  (1998),  as  well  as  behaviours  observed  in  three  or  more  instances  as 
outlined  in  Prince  and  Salas  (1999)  [reviewing  research  by  the  U.S.  Army  Aviation  Centre 
(1992)  and  Prince  and  Salas  (1993)].  These  behaviours  are  outlined  in  Table  2.  The  resulting 
list  of  behaviours  developed  for  the  purposes  of  this  evaluation  methodology  was  placed 
into  a  checksheet  format  for  systematic  use  by  the  observation  team.  The  frequency  of 
occurrence  of  these  behaviours  would  be  observed  in  order  to  ascertain  whether  there  may 
be  a  relationship  between  the  levels  of  behaviour,  specific  types  of  incidents  occurring 
during  mission  conduct  (and  their  outcomes),  and  the  potential  for  automated  tools  to 
contribute  to  the  efficiency  of  these  behavioural  processes. 

As  the  list  is  a  'first  cut',  it  will  need  to  be  refined  and  timed  to  make  it  optimally  useful  in 
terms  of  evaluating  SA  levels  and  relating  these  to  outcomes  (eg.  such  as  success  of 
mission,  achievement  of  commander's  intent).  The  list  is  shown  in  Table  3.  Additional 
consideration  will  need  to  be  given  to  the  potential  differences  between  laboratory 
controlled  simulation  experiments/ exercises,  and  field  training  exercises.  The  individual 
behaviours  have  been  placed  into  slightly  more  general  categories  (shown  in  the  left-hand 
column)  to  facilitate  ease  and  consistency  of  observations  by  the  observers. 

As  part  of  the  tuning  process,  behaviours  that  tend  to  show  constant  levels  across  different 
missions  or  experimental  scenarios  will  be  omitted,  as  observations  of  a  constantly 
occurring  behaviour  will  not  inform  the  researcher  about  the  relationship  of  the  behaviour 
to  the  mission  (or  other)  outcomes.  The  behaviour  should  not  be  completely  disregarded, 
however  it  may  remain  a  crucial  component  of  the  process  of  developing  and  maintaining 
SA.  It  is  simply  not  observed  on  a  frequency  basis  as  the  other  behaviours  are. 
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Table  1:  Behavioural  indicators  of  team  situational  awareness  (from  Table  1  in  Muniz  et  al, 
1998:p.  11-4). 

Demonstrated  awareness  of  surrounding  environment _ 

Monitored  environment  for  changes,  trends,  abnormal  conditions 

Demonstrated  awareness  of  where  he/she  was _ 

Recognised  problems _ _ _ 

Reported  problems 
Located  potential  sources  of  problems 
Demonstrated  knowledge  of  problem  consequences 

Resolved  discrepancies  _ _ 

Anticipated  a  need  for  action _ _ _ _ 

Recognised  a  need  for  action 
Anticipated  consequences  of  actions  and  decisions 
Informed  others  of  actions  taken 

Monitored  others  (self  &  others) _ _ _ 

Demonstrated  knowledge  of  tasks _ 

Demonstrated  knowledge  of  tasks 
Exhibited  skilled  time  sharing  attention  among  tasks 
Monitored  workload  (self  &  others) 

Shared  workload  within  workstation 

Answered  questions  promptly  _ 

Demonstrated  awareness  of  information _ _ 

Communicated  important  information 
Confirmed  information  when  possible 
Challenged  information  when  doubtful 
Re-checked  old  information 
Provided  information  in  advance 
Obtained  information  on  what  is  happening 
Demonstrated  understanding  of  complex  relationships 
Brief  status  frequently _ 


Table  2:  Specific  behaviours  for  rating  team  situation  awareness  (adapted  from  Prince  &  Salas, 
1999:  p.  204). 


Mission  Situational  Awareness  (U.S.  Army 
Aviation  Centre,  1992) 

Situation  Awareness  (Prince  &  Salas, 
1993) 

1.  Routinely  update  one  another  on  mission 
status  and  SA  elements'  status 

2.  Anticipate  the  SA  needs  of  others 

3.  Verbalise  and  acknowledge  changes  in 
elements  of  SA 

4.  Is  aware  of  physical  and  mental  state  of 
others 

5.  Alert  others  to  personal  problems 

6.  Alert  one  another  to  the  presence  of 
obstacles 

7.  Request  needed  information 

1.  Demonstrates  ongoing  awareness  of 
mission  status 

2.  Provides  information  in  advance 

3.  Comments  on  deviations 

4.  Demonstrates  awareness  of  task 
performance  of  self  and  others 

5.  Identifies  problems/ potential  problems 

6.  Verbalises  a  course  of  action 

DSTO-TR-1521 


Table  3:  First  cut  behaviour  list  for  developing  and  maintaining  shared  SA  in  and  between  teams. 


Checksheet  Behaviour  Category 

Behaviours  included  under  the  category 

Monitor  environment /battle 

•  Monitor  environment 

•  Monitor  visualisation  screens 

Monitor  others 

•  Watch  others  at  work 

State  location 

•  State  location  of  self  or  others 

Request  information 

•  Request  information  on  current  situation,  status, 
location,  weather,  friendly,  enemy,  civilian,  etc 

Receive  information 

•  Receive  information  on  current  situation,  status, 
location,  etc 

Send  information 

•  Send  information  via  computer  terminals 

•  Send  information  manually  (paper) 

•  Send  information  via  radio 

Indicate  information  on  map 

•  Mark  known  routes/locations  of  entities 

•  Mark  anticipated  routes /locations 

•  Mark  other  information  on  map 

Explain  information 

•  Explain  information  to  others 

Confirm  information 

•  Confirm  new  information 

•  Recheck  old  information 

•  Challenge  information 

Anticipate  need  for  information 

•  Provide  information  prior  to  request 

•  Provide  advice/instructions/explanation  prior  to 
request 

Brief  status 

•  Brief  team  members  on  current  status,  locations, 
etc 

Discuss  task  knowledge 

•  Discuss  task(s) 

•  State  understanding  of  complex  relationships 

Identify  discrepancies  /deviations 
from  plan 

•  Identify  and  discuss  deviations  or  discrepancies 
from  plan 

Record  deviations  from  plan 

•  Record  deviations  from  original  plan 

Report  problems 

•  Report  problems  to  others 

•  Discuss  consequences  of  problems 

•  Indicate  sources  of  problems 

Discuss  actions 

•  State/discuss  need  for  action 

•  Discuss  possible  outcomes  of  actions 

Inform  others  of  decision/actions 

•  State  decision/  action  information  to  others 

Implement  remedial  action 

•  Implement  action(s)  to  deal  with  problems 

•  Implement  change  of  plan/  action 

Reallocate  workload 

•  Reassign  work  to  team  members 

As  Tables  1,  2  and  3  show,  the  majority  of  behaviours  crucial  to  SA  involve 
communication  and  the  handling  of  information.  Information  is,  it  has  been  argued,  the 
critical  component  of  military  HQ  operations.  In  particular,  information  regarding  identity, 
location,  status,  and  intent  with  regard  to  (i)  own  forces,  (ii)  enemy  forces,  and  (iii)  neutral 
forces,  as  well  as  the  topography  and  the  weather  are  the  basis  of  the  Commander's  Critical 
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Information  Requirements  (CCIR)  (Rees  and  Bowden,  in  process).  These  appear  to  be 
common  to  all  military  headquarters  (HQs),  and  are  vital  to  the  understanding  and 
interpretation  of  the  battlespace.  Thus,  these  information  categories  essentially  define  the 
data  required  to  obtain  situation  awareness  in  the  battlespace  (Rees  and  Bowden,  in 
process). 

Given  the  reliance  of  HQs  on  obtaining,  interpreting,  and  applying  battlespace 
information,  any  attempt  to  design  and  introduce  automated  tools  to  this  arena  should 
take  these  information  needs  into  consideration.  This  then  begs  the  question  of  the  ability 
of  the  introduced  tools  to  facilitate  the  transfer,  handling,  and  interpretation/ 
understanding  of  these  pieces  of  information  by  the  commander  and  his  staff. 

Developing  a  series  of  accurate  and  consistent  behavioural  indicators  of  performance  may 
be  crucial  to  the  understanding  of  the  impact  of  tools  and  technologies  on  the  performance 
of  the  military;  in  addition,  the  use  of  tools  and  technologically  enhanced  systems  may 
produce  specific  training  needs.  These  can  potentially  be  identified  and  understood  within 
the  framework  of  a  behavioural  foundation.  Using  a  'behaviour  systems'  type  approach — 
pioneered  by  Timberlake  (1983;  see  also  1993,  and  Timberlake  and  Silva,  1995)  in  the  field 
of  animal  behaviour,  yet  equally  applicable  to  human  behaviour  —  it  may  be  possible  to 
develop  a  picture  of  the  activities  that  military  subjects  perform  and  the  possible  effects  of 
changes  to  their  environment  (including  the  tools  available  for  their  use).  This  stems  from 
the  use  of  the  systems  perspective  in  the  study  of  human  behaviour  and  outcomes. 

The  expression  of  behaviours  and  activities  from  a  systems  perspective  is  elegant  and 
relatively  straightforward.  An  example  of  an  animal  behaviour  system  and  a  simple 
military  example  can  be  found  in  Appendix  A.  Essentially,  behaviour  systems  express 
behaviour  in  terms  of  several  levels  or  groupings  of  behaviour  types,  that  is,  the  system  the 
organism  (in  this  case,  a  human)  is  functioning  within.  In  terms  of  animal  behaviour,  this 
could  be  something  such  as  feeding.  For  military  purposes,  it  could  be  something  such  as 
'planning'.  The  individual  components  of  a  system  are  functional  units  that  represent 
typical  combinations  of  determinants  and  categories  of  outcome.  The  four  levels  of  control 
hierarchy  outlined  in  this  paragraph  —  and  their  components  — select  and  coordinate 
individual  responses  (termed  action  patterns).  The  system  level  accounts  for  the  tendency 
of  behaviour  to  be  organised  around  important  functions.  Below  this  is  a  subsystem,  which 
indicates  the  sub-category  into  which  the  behaviour  group  fits.  An  example  here  might  be 
'mission  planning'  or  'Strategic  planning',  for  example.  Subsystems  refer  to  combinations  of 
stimulus  sensitivities,  motivational  states,  and  response  components  that  constitute 
strategies  for  meeting  the  needs  of  any  system.  The  activity  of  a  subsystem,  then,  should 
increase  a  subject's  sensitivity  to  certain  stimuli  and  therefore  increase  the  probability  that 
the  relevant  modes  and  modules  will  be  expressed  as  coherent  sets  of  action  patterns. 

Within  these  subsystems,  there  are  other  more  specific  groupings  called  'modes',  which 
denote  motivational  substates  that  relate  to  the  sequential  and  temporal  organisation  of 
action  patterns.  Within  planning,  for  example,  these  may  include  'Mission  Analysis', 
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'Course  of  Action  (CO A)  Development',  COA  Analysis',  and  'Decision  and  Execution'4. 
Perceptual-motor  modules  (hereafter  termed  'modules')  are  the  next  level  down,  and  these 
indicate  the  predisposition  to  respond  to  certain  stimuli  with  particular  response 
components.  The  perceptual  aspects  include  sensory  filters  that  modify  incoming  stimulus 
properties  and  assist  in  organising  and  combining  different  stimuli.  Motor  organisation 
refers  to  motor  programs  or  their  assembly.  Action  patterns  (hereafter  referred  to  as 
'actions’)  are  coherent  sets  of  behavioural  responses. 

Clearly,  the  animal  and  human  behaviour  systems  are  action  oriented  (that  is,  actions  are 
performed  to  achieve  the  higher  goals  of  the  system),  however  the  human  system  tends 
more  towards  the  handling  and  utilisation  of  information. 

According  to  the  systems  perspective,  humans  are  a  reliable  'stand  alone'  system,  and  the 
commission  of  errors  in  the  work  setting  (for  example)  tend  to  arise  when  humans  are 
interacting  with  technological  systems.  Serig  (2002)  states  that  these  errors  can  be  triggered 
by  technology  and  its  environment,  as  a  result  of  the  way  these  factors  interact  and 
challenge  human  limitations.  Here,  we  can  see  that  there  is  a  worthwhile  avenue  of 
research  that  may  benefit  both  the  military  and  the  organisations  attempting  to  provide 
technologies  to  enhance  the  performance  of  the  military.  It  has  been  observed  that  there  is 
ample  opportunity  for  errors  to  occur  during  the  operation  of  new  or  relatively  unfamiliar 
technology  in  most  settings,  and  clearly  if  this  is  occurring  in  the  military  operational 
setting,  the  results  could  be  disastrous.  Benefit  can  be  seen,  then,  in  conducting  training 
exercises  and  experiments  in  order  to  discover  the  potential  areas  for  error  before  they 
become  an  operational  issue.  This  will  be  further  explored  in  future  studies. 

1.2  The  observational  data  collection  methodology 

In  order  to  generate  the  type  of  data  necessary  for  this  type  of  behavioural  evaluation  and 
relation  of  behaviours  to  events,  outcomes  and  tool  performance,  the  observational  data 
collection  methodology  (ODCM)  was  implemented.  This  is  briefly  described  below. 

Observational  data  collection  functions  as  an  objective  and  unobtrusive  way  of  recording 
events /behaviours,  which  can  then  be  used  for  evaluating  team  performance  in,  eg.  an 
HQ.  It  is  unobtrusive  because  it  does  not  involve  interrupting  the  workings  of  the  team  in 
any  way,  and  objective  because  it  does  not  involve  recording  opinions  or  interpretations  of 
what  is  occurring,  simply  the  behaviour(s)  observed  at  any  given  time. 

This  method  is  excellent  for  use  in  situations  where  interference  with  participants  is  not 
feasible  (ie.  would  cause  possible  danger  to  the  participants)  or  where  it  would  affect  the 
outcomes  of  the  experiment  such  that  the  resulting  data  would  not  be  meaningful.  Is 
possibly  more  reliable  than  subjective  data  in  some  ways,  because  there  is  no  'after-the- 
fact'  bias  or  tainted  recall  of  events  or  actions. 


4  In  keeping  with  the  military  appreciation  process  (MAP)  structure. 
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Initially,  a  taxonomy  of  behaviour  should  be  designed  via: 

(i)  working  from  the  literature  in  the  area  (previous  experimental  observations, 
field  trial  observations,  SOPs  and  manuals,  pamphlets,  etc) 

(ii)  preliminary  observations  (continuous,  with  note  taking  where  this  is  possible 
well  prior  to  experimental  observations),  and  —  in  some  cases  — 

(iii)  interviews  with  military  staff 

This  enables  the  researcher  to  build  a  holistic  picture  of  the  behaviours  occurring  within 
the  chosen  context,  and  should  provide  an  overall  view  of  the  range  of  behaviours 
performed.  Within  this  behaviour  range,  there  may  be  a  behavioural  subset  that  will  be 
useful  in  terms  of  evaluating  the  effects  of  introduced  tools  or  techniques  on  specific  team 
functioning.  This  will  be  determined  following  problem  definition. 

Problem  definition  is  vital  to  good  data  collection.  It  allows  researchers  to  focus  their 
observations  and  collect  only  the  data  necessary  to  answer  the  question  at  hand.  It  is 
impossible  to  observe  everything  that  occurs,  and  observers  should  not  attempt  this  if  they 
want  to  record  accurate  and  meaningful  data.  Thus,  a  project  should  have  a  well-defined 
aim  and  set  of  objectives,  and  the  data  gathered  should  attempt  to  address  these.  For 
example,  a  researcher  may  be  concerned  with  how  the  Int  cell  of  a  HQ  performs,  and  what 
effect  introducing  a  new  communication  medium  will  have  on  this  performance.  Here,  an 
overall  picture  of  Int  cell  functioning  should  be  developed  first,  and  then  observations 
taken  again  once  the  new  communication  medium  is  introduced.  These  observations  can 
be  compared  with  those  during  baseline  functioning  in  order  to  provide  an  assessment  of 
effect. 

In  order  to  do  this  systematically,  the  variables  being  examined  should  also  be  determined 
beforehand.  These  variables  will  also  emerge  from  definition  of  the  problem.  That  is,  there 
are  independent  and  dependent  variables  that  must  be  outlined  prior  to  beginning  data 
collection.  Independent  variables  are  also  known  as  factors  —  in  the  case  of  the  above 
example,  these  would  be  the  new  communication  medium.  The  dependent  variables  are  the 
'measurables'  (ie.  those  whose  levels  are  affected  by  the  factors,  and  which  can  be 
observed  or  measured  in  some  way).  Dependent  variables  can  therefore  provide  an 
indication  of  the  affects  of  independent  variables  on  the  performance  of  a  team.  So  in  the 
case  of  the  Int  cell  example,  the  independent  variable  would  be  the  new  communication 
medium,  with  the  dependent  variables  being  the  team  performance  attributes  (such  as 
communication,  cognitive  and  physical  workloads,  and  frustration).5 


5  Refer  to  the  technical  report  (DSTO-TR- 1 034)  on  research  methodology  by  Mills  and  Stothard  (2000)  for  a 
more  detailed  explanation. 
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1.3  The  relationship  between  behavioural  SA  indicators  and  measures 
of  performance 

Behavioural  indicators  of  SA  alone  are  not  enough  to  perform  an  evaluation  of  team 
performance.  They  must  be  tied  in  closely  with  outcomes  provided  by  such  measures  as 
(for  example): 

a)  how  well  the  commander's  intent  was  met 

b)  the  number  of  enemy  destroyed 

c)  the  timeliness  of  actions  by  various  parts  of  the  overall  human-machine  system 

d)  the  ratio  of  enemy  kills  to  friendly  losses 

e)  the  length  of  time  blue  assets  were  vulnerable  to  enemy  weaponry 

This  will  produce  an  evaluation  method  with  much  more  utility  than  a  behavioural 
assessment  alone. 

The  aims  of  this  paper  are  as  follows: 

•  to  illustrate  the  operation  of  an  HQ  through  observing  the  behaviour  of  staff,  and 

•  to  attempt  to  relate  behavioural  and  outcome  measures  to  provide  a  meaningful 
evaluation  of  team  performance,  with  particular  reference  to  SA  (this  is  important,  as 
SA  is  an  enabler  for  the  effective  functioning  of  the  forces  in  the  battlespace). 

Additional  benefits  of  designing  a  behaviour  taxonomy  should  emerge  as  the  tools  are  put 
in  place  and  used  by  HQ  staff.  That  is,  such  a  taxonomy  of  behaviour  should  provide  a 
reference  for  the  effects  of  adding  automated  tools  into  the  human-machine  system.  That 
is,  questions  need  to  be  answered  with  respect  to: 

•  how  these  tools  affect  team  performance  overall 

•  where  the  initial  effect(s)  occur  (ie.  is  there  a  specific  point  in  the  behavioural 
repertoire  which  has  been  altered) 

This  type  of  analysis  should  assist  in  providing  guidance  for  tool  redesign  and  training 
needs  analyses  if  the  performance  of  the  team  is  adversely  affected  by  the  addition  of  the 
new  tools6. 


2.  Data  Collection 

2.1  Exercise  Prowling  Pegasus 

Prowling  Pegasus  was  run  from  March  19th  to  22nd,  2001  at  DSTO  Edinburgh,  with  a 
connection  to  AOD  at  DSTO  Melbourne.  The  main  aim  of  this  exercise  was  to  support 
ARH  doctrine  and  TTP  development,  as  well  as  the  development  of  Joint  Land  Air 
Systems  concepts. 


6  Following  an  adequate  training  and  familiarisation  period,  that  is.  Initial  adverse  effects  may 
simply  be  due  to  the  unfamiliarity  with  tools  and  technology,  and  may  be  mitigated  by  training. 
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Relating  to  this  aim  were  a  number  of  objectives  including: 

•  Examining  the  impact  of  the  new  battlespace  visualisation  and  management  tools 
on  procedures 

•  Determination  of  joint  Land  Air  System  teams  effectiveness  in  meeting  the 
commander's  intent 

•  Examining  the  behaviours  associated  with  the  use  of  the  new  situation  awareness 
tools 

The  experiment  involved  several  key  aspects:  (1)  a  large  number  of  military  and  DSTO 
civilian  research  personnel;  (2)  a  large  number  of  technical  staff  responsible  for  the 
connectivity  of  the  variety  of  systems  and  tools  being  utilised  in  the  experiment,  as  well  as 
video  and  sound  recording  of  the  experimental  proceedings;  (3)  a  multitude  of  hardware 
and  software  systems  on  site  at  DSTO  Edinburgh;  and  (4)  a  connection  to  AOD  in 
Melbourne  for  video  conferencing,  input  into  the  AARs7,  and  Fast  Air  input  into  the 
scenarios. 

2.1.1  Experimental  structure 

There  were  four  days  of  experimentation,  during  which  a  single  scenario  was  run  multiple 
times  under  different  conditions.  That  is,  each  scenario  was  a  repeat  of  the  previous  one 
but  utilising  different  combinations  of  assets  to  examine  the  effectiveness  of  teaming  (or 
the  effect  of  various  force  mixes  on  the  outcomes  of  missions). 

The  first  day  was  designed  to  familiarise  the  military  participants  with  both  the  operation 
of  the  Land  Situation  Awareness  System  (LSAS)  in  terms  of  its  capabilities  and  the  manner 
in  which  functions  would  be  carried  out,  and  with  the  background  to  each  of  the  four 
missions. 

Day  two  saw  the  first  mission  (Ml)  as  a  practice  mission,  with  last  minute  glitches  and 
system  unfamiliarities  being  attended  to  prior  to  the  three  experimental  runs  of  the 
scenario.  It  also  gave  the  military  staff  an  opportunity  to  familiarise  themselves  with  the 
LSAS  terminal  operators  they  would  be  working  with  for  the  next  three  missions. 

Missions  two  to  four  (M2  to  M4)  were  those  during  which  the  observers  collected  the  bulk 
of  their  data,  and  these  lasted  approximately  1.5  hours  to  2  hours  each.  Each  of  the 
missions  involved  the  same  scenario,  however  the  configuration  of  assets  varied  according 
to  the  list  below. 

Mission  1:  The  two  ARHs  conducting  search  and  destroy  missions.  The  helos  were 
networked  to  each  other  as  well  as  to  the  ground-based  HQ. 


7  After  Action  Reviews.  These  were  designed  as  scientific  debriefs  rather  than  formal  military 
AARs  in  order  to  supplement  the  data  collected  during  the  missions  themselves. 


9 


DSTO-TR-1521 


Mission  2:  As  for  Ml,  but  with  the  two  ARHs  coordinating  with  (a)  Special  forces 
conducting  close  reconnaissance  and  target  designation;  (b)  GBAD  for  airspace 
control;  and  (c)  UAV  for  reconnaissance. 

Mission  3:  As  for  M2,  but  with  fixed  wing  added  and  the  ARHs  acting  as  forward 
air  controllers. 

Mission  4:  As  for  M3,  but  with  the  ARHs  also  carrying  out  an  attack  function. 

The  scenario  developed  to  explore  the  potential  of  these  teaming  arrangements  revolved 
around  the  Land  Air  System  (LAS)  assisting  a  conventional  mechanised  Brigade  (Bde)  in 
an  operation  to  expel  enemy  forces  from  a  town  (in  this  case,  the  town  of  Katherine  in  the 
Northern  Territory,  Australia).  Priority  enemy  targets  were  to  be  destroyed  outside  the 
town  limits.  The  missions  relied  on  the  operation  of  a  synthetic  environment  (SE), 
described  in  Seymour,  Sands,  Grisogono,  Unewisse,  Vaughan  and  Baumgart  (2001: 8)  and 
paraphrased  in  section  2.1.2. 

2.1.2  The  Synthetic  Environment 

The  System  Concept  Demonstrator  (SCD)  of  the  Land- Air  battle  group  (BG)  interacted 
with  constructive  simulations  of  the  enemy  force  and  the  remainder  of  the  friendly  forces. 
Additional  Land  components  were  simulated  using  ModSAF,  while  the  extra  Air  input 
was  provided  by  the  STAGE  simulation.  Models  of  radar  surveillance  were  also  included, 
and  the  entire  SE  was  linked  using  DIS  protocols.  The  information  coming  from  the 
constructive  simulations  included  the  position  and  status  of  all  the  blue  and  red  elements 
(where  the  red  element  information  was  determined  by  the  available  surveillance  assets). 
Where  surveillance  assets  were  modelling  in  ModSAF,  automatic  feeds  of  information 
went  into  the  information  space  (infospace).  Where  humans  detected  and  identified  a 
virtual  entity  through  their  view  into  the  virtual  world,  the  information  was  manually 
entered  into  the  infospace  via  the  LSAS  interface  (and  an  experienced  operator).  The 
movements  of  the  blue  entities  were  carried  out  by  human  operators  (Locon)  who  received 
voice  or  LSAS  carried  commands  from  the  HQ  (often  a  combination  of  these  two).  A  DSTO 
staff  member  chosen  to  represent  the  enemy  commander  controlled  the  red  forces. 

2.2  Subjects 

The  subject  of  observations  were  the  military  participants  in  the  Prowling  Pegasus 
exercise,  specifically  those  staffing  the  Bde  HQ,  JOSCC,  Avn  Regimental  HQ  and  Locon. 
This  was  a  maximum  of  15  people  at  any  given  time  during  the  observation  periods. 
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2.3  Observation  Methods 

2.3.1  The  observation  method  for  Ex  Prowling  Pegasus 

Of  a  team  of  four  observers,  one  individual  was  located  at  each  of  the  main  HQ  areas  (Bde 
HQ,  JOSCC,  Regimental  HQ,  and  Locon)  in  order  to  observe  the  behaviours  of  the 
participants  relating  to  situation  awareness  development  and  maintenance  and  use  of  the 
Land  SA  and  other  tools  for  the  duration  of  each  mission.  The  observers  were  equipped 
with  SA  checksheets.  Tool  Use  checksheets,  folders,  notebooks  for  opportunistic 
observations  and  general  notes,  and  pencils. 

The  checksheets  for  SA  and  Tool  Use  are  shown  in  Appendix  B. 

Observations  were  taken  using  both  the  continuous  sampling  method  (ie.  observers  record 
each  instance  of  the  listed  behaviours  in  the  box  corresponding  to  the  appropriate  time 
interval)  and  the  instantaneous  scan  sample  (ie.  a  scan  of  the  team  is  done  at  the  end  of 
each  time  interval  and  the  behaviour  occurring  at  the  time  is  recorded).  Thus,  the  SA 
checksheet  provides  a  frequency  of  behaviour  over  time,  while  the  Tool  Use  checksheet 
provides  a  representation  of  the  overall  average  level  of  tool  use. 


3.  Results 

Note  that  results  for  mission  1  are  not  presented,  as  the  data  collected  were  too  unreliable 
due  to  the  'practice'  type  nature  of  this  initial  scenario  run.  Thus,  all  results  will  refer  to 
Missions  2, 3  and  4. 

3.1  The  SA-related  behaviour  list 

Table  4  presents  the  behaviours  recorded  by  observers  in  each  location  during  Ex 
Prowling  Pegasus  by  mission.  Behaviours  or  categories  which  were  not  observed  six  or 
more  times  out  of  a  possible  ten  observations  (that  is,  in  three  main  locations  across  the 
three  main  missions,  and  also  in  Locon  during  mission  4)  will  be  considered  for  omission 
from  the  active  behaviour  list.  These  behaviour  categories  include: 

(1)  Monitor  others 

(2)  State  location 

(3)  Reallocate  workload 

The  lack  of  observation  of  some  of  these  behaviours  can  be  explained  by  some  of  the 
circumstances  of  the  experiment.  Monitor  others  was  not  observed  in  the  Bde  HQ  or  the 
Avn  Regimental  HQ,  possibly  due  to  the  fact  that  both  of  these  locations  contained  only 
two  staff  members.  The  JOSCC  and  Locon  both  observed  the  monitor  others  behaviour 
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category  at  relatively  high  levels,  and  the  number  of  people  in  each  of  these  locations  was 
generally  a  minimum  of  four. 


Table  4:  Missions  in  which  situation  awareness  related  behaviours  were  not  observed  for  each 
location  are  indicated  by  a  cross  in  the  table  below.  Note  that  LoCon  is  omitted  here  because  all 
categories  of  behaviour  were  observed  in  that  location.  Ratio  0:N  indicates  the  ratio  of locations  in 
which  the  behaviour  was  observed  versus  not  observed. 


Location» 

Bde 

HQ 

■ 

■ 

JOSCC 

■ 

■ 

Avn 

Reg 

HQ 

■ 

■ 

Ratio 

(Q:N) 

Mission»> 

M2 

M3 

M4 

M2 

M3 

M4 

M2 

M3 

M4 

Behaviour: 

Monitor  others 

X 

X 

X 

X 

X 

X 

■ 

State  location 

X 

X 

X 

X 

X 

X 

Explain  info 

X 

8:1 

Anticipate  need  for  info 

X 

X 

X 

X 

X 

X 

X 

1:8 

Indicate  info  on  map 

X 

X 

X 

6:3 

Identify  deviations  from  plan 

X 

X 

X 

X 

X 

Record  deviations  from  plan 

X 

X 

X 

X 

X 

BBI1M 

Report  problems 

X 

X 

X 

X 

X 

■ 

■ 

■ 

Inform  others  of  action/ 
decision 

X 

X 

X 

HI 

■ 

■ 

6:3 

Implement  remedial  actions 

X 

X 

X 

X 

X 

4:5 

Re  -allocate  workload 

X 

X 

X 

X 

X 

X 

X 

X 

1:8 

Thus,  referring  to  Table  4,  there  are  a  number  of  behaviours  that  will  potentially  be 
excluded  from  the  observational  checksheet  in  future  data  collection  for  various  reasons, 
including: 

•  Lack  of  applicability  to  the  situation  under  observation 

•  Constancy  of  the  level  of  the  behaviour  across  situations 

•  The  behaviour  not  being  recorded  by  observers  during  the  course  of  the 
experiment 

In  addition,  there  are  issues  in  terms  of  the  overall  levels  at  which  behaviours  were 
observed  during  the  exercise.  That  is,  there  are  clear  differences  in  the  levels  at  which 
certain  behaviours  were  seen  to  occur  during  the  225  minutes  of  mission  time  (for  missions 
2, 3  and  4  as  analysed  in  this  report).  Some  behaviours  occur  at  very  low  levels,  and  it  is 
unlikely  that  these  will  have  a  large  impact  on  overall  performance.  It  may  be  that  the 
behaviours  are  not  carried  out  overtly:  that  is,  there  may  be  unspoken  or  automatic 
tendencies  to  do  certain  tasks  which  make  them  difficult  to  observe.  Table  5  shows  the 
overall  levels  of  behaviour  across  the  three  missions  and  all  locations. 
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Table  5:  The  total  occurrences  of  behaviour  during  missions  2,3  and  4  across  locations 


Behaviour 

Total  Occurrences 

Request  information 

159 

Receive  information 

345 

Send  information 

325 

Indicate  information  on  map 

200 

Explain  information 

120 

Confirm  information 

76 

Anticipate  information  needs  of  others 

13 

Brief  status 

201 

Identify  deviations  from  plan 

6 

Record  deviations  from  plan 

28 

Report  problems 

14 

Discuss  actions /decisions 

485 

Inform  others  of  actions/decisions 

128 

Implement  remedial  actions 

14 

Reallocate  workload 

2 

Total  Observation  Time:  225  mins 

Clearly,  behaviours  such  as  identify  deviations  from  plan,  record  deviations  from  plan, 
implement  remedial  actions,  report  problems,  and  reallocate  workload  occur  at  very  low  levels.  In 
addition,  anticipate  information  needs  of  others  also  occurs  at  a  low  level  here,  but  this  must 
be  interpreted  with  care.  Previous  observations  of  teams  with  higher  levels  of  experience 
working  together  indicate  that  the  level  of  this  behaviour  can  be  expected  to  rise  as  the 
team  members  become  more  familiar  with  each  other.  During  observations  conducted  at 
the  Headline  2000  experiment,  the  Bde  HQ  team  showed  higher  levels  of  this  type  of  pre¬ 
empting  behaviour  than  the  BG  HQ  team  (Mills,  Huf  and  Kardos,  2001).  In  this  scenario, 
the  Bde  HQ  team  members  had  worked  together  on  more  occasions  than  the  BG  HQ  team, 
leading  to  greater  familiarity.  In  all  cases,  the  artificiality  of  the  experimental  setup  — 
particularly  in  terms  of  team  structure  and  size  —  must  be  considered  before  conclusions 
are  drawn  too  hastily. 

Thus,  although  there  is  no  doubt  that  the  behaviour  list  will  need  to  be  modified;  the 
reasons  for  the  omission  of  certain  behaviours  during  the  exercise  must  be  considered 
carefully.  It  is  vital  that  some  important  aspect  of  team  interaction  or  team  member 
behaviour  which  contributes  to  SA  development  and  maintenance  is  not  prematurely 
omitted  on  the  basis  of  limited  evidence.  Some  of  the  potential  reasons  for  particular 
behaviours  not  being  observed  in  this  situation  may  include: 

•  Moderate  to  low  fidelity  compared  with  real  HQ  staffing  setups  (very  few  staff 
imitating  the  functions  usually  carried  out  by  far  more  individuals,  and  lack  of  an 
S2  in  the  Bde  HQ) 

•  The  lack  of  familiarity  of  the  military  with  their  assigned  team  members 

•  Lack  of  experience  of  the  observers  leading  to  inability  to  identify,  or  inconsistency 
in  identifying  the  behaviours  in  question 


13 


DSTO-TR-1521 


•  Lack  of  equipment  to  enable  the  behaviour  to  occur  (eg.  Avn  Regimental  HQ  were 
given  the  ModSAF  view  only  during  mission  4) 

It  is  also  possible  that  the  low  level  behaviours  themselves  may  need  to  be  altered  to  better 
reflect  what  is  going  on  in  the  HQs. 

3.2  Levels  of  behaviour 

For  ease  of  presentation  of  the  results,  the  behaviours  are  divided  into  categories  relating 
to  their  functions.  The  first  category  is  Information  Handling,  and  involves  behaviours  that 
move  information  to,  from,  and  between  staff  within  the  various  HQs,  as  well  as  to  the 
units  in  the  field.  The  levels  of  behaviour  relating  to  Information  Handling  during  mission 
two  for  each  location  are  shown  in  Figures  1  to  3.  The  behaviour  levels  are  presented  as 
percentage  contributions  to  the  total  in  each  location,  with  time  intervals  indicating  five 
minutes  each. 


Figure  1:  Bde  HQ  information-related  SA  behaviour  during  mission  2 


Time  block 


10  Request  ■  Receive  □  Send  □  Indicate  on  mad 

Figure  2:  fOSCC  information-related  SA  behaviour  during  mission  2 

Clearly,  the  Bde  HQ  in  Figure  1  shows  a  high  tendency  towards  requesting  and  receiving 
information,  with  an  increased  tendency  towards  indicating  information  on  maps  near  the  end 
of  the  mission.  The  JOSCC  shown  in  Figure  2,  by  contrast,  shows  moderate  levels  of  both 
requesting  and  receiving  information,  with  much  higher  levels  of  explaining  information 
obvious  throughout  the  mission.  In  Figure  3,  the  high  levels  of  the  behaviours  receive  and 
send  information,  along  with  sporadically  moderate  levels  of  both  request  and  confirm 
information  supports  the  fact  that  the  Avn  Reg  HQ  tended  to  depend  more  on  the  radio 
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communications  than  the  SA  tool  (computer  screen)  for  its  SA  updates.  The  Avn  Reg  HQ 
staff  also  show  a  level  of  anticipatory  behaviour  not  encountered  in  the  other  locations.  This 
may  be  due  to  the  ability  of  a  smaller  number  of  staff  to  acclimatise  to  the  needs  of  their 
fewer  co-workers  more  quickly  (cf  a  large  group). 


Figure  3:  Avn  Reg  HQ  information-related  SA  behaviour  during  mission  2 

When  the  critical  events  list  is  examined,  it  is  apparent  that  there  are  peaks  in  particular 
behaviours  during  or  adjacent  to  time  intervals  containing  critical  events.  During  time 
interval  5,  for  instance,  Bde  HQ  issued  orders  to  the  elements  engaging  the  T-80s  to  the 
effect  that  these  elements  were  to  withdraw  and  allow  the  tanks  to  be  engaged  by  the 
helicopters  instead.  Figure  1  shows  that  this  coincides  with  high  levels  of  receiving  and 
explaining  information,  and  indicating  information  on  the  map.  In  the  JOSCC,  however,  this 
time  interval  coincides  with  a  high  level  of  explain  information,  concurrent  with  moderate 
levels  of  receive  information,  and  low  levels  of  confirm  information  and  indicate  information  on 
map.  During  this  period,  the  Avn  Reg  HQ  demonstrates  very  high  levels  receive  information, 
with  moderate  levels  of  request,  send  and  anticipate  information  and  moderate  levels  of 
request  and  send  information  apparent.  These  levels  of  behaviour  indicate  that  something  of 
importance  was  occurring  during  this  time  interval,  and  that  it  was  of  particular 
importance  for  the  Avn  component  of  the  blue  force.  The  higher  levels  of  send  information 
occurring  in  the  following  time  interval  indicates  that  the  information  received  by  (and 
processed  in)  the  Avn  Reg  HQ  was  then  imparted  to  other  locations  (Bde  HQ,  for 
example)  in  order  to  maintain  the  SA  of  the  team  members. 

The  second  category  of  behaviour  used  here  is  the  Status/ Action  category.  The  average 
levels  of  behaviour  relating  to  commander's  perception  of  the  current  situation  and 
necessary  actions  (during  mission  2)  are  shown  in  Figures  4  to  6.  Again,  the  time  intervals 
shown  indicate  five  minutes  each. 

When  Figures  4, 5  and  6  are  compared  with  the  table  of  critical  events  for  mission  2  (Table 
6),  it  is  clear  that  during  time  interval  5  the  Bde  HQ  focus  is  on  briefing  the  status  of  the 
mission  to  the  staff,  while  the  JOSCC  shows  no  behaviour  during  that  period  (which 
relates  to  the  staff  attending  to  the  Bde  HQ  briefing).  The  Avn  Reg  HQ  shows  high  levels 
of  brief  status  and  discuss  actions/decisions,  with  a  moderate  level  of  inform  others  of 
actions/decisions  during  this  interval. 
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100% 

80% 
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20% 

0% 

1  2  3  4  5  6  7  8  9  10  11  12  13  14  15 

Time  block 


■  Brief  status  ■  Discuss  actions/decisions 

■  hform others  of  actions/decisions  _ _ 

Figure  4:  Status  and  action-related  behaviour  levels  in  the  Bde  HQ  during  mission  2 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15 

Time  block 


■  Brief  status  ■  Discuss  actions/decisions 

■  Inform  others  of  actions/decisions _ _  J 

Figure  5:  Status  and  action-related  behaviour  in  the  JOSCC  during  mission  2 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15 

Time  block 


■  Brief  status  ■  Discuss  actions/decisions 

■  hform  others  of  actions/decisions  _ _ _  I 

Figure  6:  Status  and  action-related  behaviour  levels  in  the  Avn  Reg  HQ  during  mission  2 

This  supports  the  assumption  that  the  workings  of  the  airborne  component  of  the  force  are 
of  importance  at  this  point  in  time.  This  is  further  supported  by  the  rise  in  discussing 
actions/decisions  and  informing  others  of  actions/decisions  in  the  following  time  interval,  as  the 
Avn  Reg  HQ  is  ensuring  that  the  rest  of  the  team  are  aware  of  events  and  the  status  of  air 
assets. 
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Table  6:  Events  occurring  during  each  time  block  for  mission  2 


Time 

interval 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

Event 

Helo  1  crashed 

Helo  2  landing  for  replen  waitout 

ManPad  destroyed;  Helo  2  back  up 

Order  from  Bde  HQ  wrt  elements  engaging  T- 
80s  to  withdraw,  helos  to  engage  instead 

Helos  engaged  and  destroyed  T-80s;  order 
from  Bde  HQ  to  BG  Locust  re:  movement 

Weevil  reports  3  x  BTRs  destroyed;  Bde 
request  confirm  en  target  destroyed 

Helos  engage  enemy  near  burnt  out  carriers; 
en  at  objective  4  engage  blue  forces 

Identified  Coy  of  T-80s  between  GS  0796  - 
0799 

Helo  2  shot  down 

BGs  Sabre  &  Mitre  secure  Knots  Crossing; 
Manbulloo  reported  secured 

Figures  7, 8  and  9  show  the  proportions  of  information  related  behaviour  occurring  in  each 
location  during  mission  3.  It  is  clear  that  the  Avn  Reg  HQ  shows  a  much  higher  level  of 
confirmatory  behaviour  than  either  the  Bde  HQ  or  JOSCC  throughout  the  mission.  Bde  HQ 
seems  to  show  the  highest  incidence  of  requesting  information,  with  the  JOSCC  initially 
showing  a  higher  level  of  receiving  information,  which  then  turns  to  higher  levels  of 
indicating  information  on  maps  (at  time  interval  5)  and  sending  information  (at  time  interval 
9).  Comparing  the  levels  of  behaviour  in  the  figures  above  with  the  events  listed  in  Table 
6,  it  is  apparent  that  there  is  a  relationship  between  critical  occurrences  and  increases  in 
the  levels  of  particular  behaviours  in  the  different  locations.  This  will  be  further  discussed 
in  section  3.2.1,  however  some  of  the  patterns  will  be  discussed  here. 

Table  7  indicates  that  during  time  interval  7,  the  bluefor  mechanised  infantry  were 
engaged  with  enemy  T-80s,  and  that  the  helicopters  were  tasked  to  provide  offensive  fire 
support.  When  Figures  8, 9  and  10  are  examined,  it  is  clear  that  the  Bde  HQ  shows  a  high 
level  of  sending  information  during  this  time  interval,  with  moderate  levels  of  behaviours 
such  as  request,  receive  and  indicate  information  on  maps.  The  JOSCC,  by  comparison,  shows 
higher  levels  of  indicate  information  on  maps,  with  moderate  levels  of  behaviour  such  as 
request,  receive  and  confirm  information.  The  Avn  Reg  HQ  shows  very  high  levels  of  receive 
information,  with  moderate  levels  of  request,  send  and  confirm  information  shown  in  Figure  9. 
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Figure  7:  Bde  HQ  proportions  of  information  related  behaviours  during  mission  3 


Time  Block 


D  Request  ■  Receive  □  Send  □  Indicate  on  ma| 

■  Explain _ ■  Confirm _ ■  Anticipate _ 


Figure  8:  JOSCC  proportions  of  information  related  behaviours  during  mission  3 


Time  Block 


B  Request  B  Receive  □  Send  □  Indicate  on  map 

B  Explain  B  Confirm _ B  Anticipate _ _ _ 

Figure  9:  Avn  Reg  HQ  proportions  of  information  related  behaviours  during  mission  3 

Figures  10, 11  and  12  indicate  the  proportions  of  behaviours  relating  to  the  commander's 
perception  of  the  current  situation  and  necessary  actions  during  this  mission.  When  these 
figures  are  examined,  it  is  apparent  that  while  Bde  HQ  has  such  a  high  concentration  on 
sending,  requesting,  receiving  and  indicating  information  on  maps  behaviours,  there  is  no  time 
spent  on  briefing  status,  discussing  actions/decisions,  or  informing  others  of  action/decisions.  The 
JOSCC,  however,  shows  extremely  high  levels  of  discussing  actions, /decisions,  coupled  with 
moderate  levels  of  the  behaviour  brief  status.  As  the  JOSCC  has  incoming  information  from 
both  the  mechanised  units  and  the  aviation  components  of  the  force,  it  is  the  hub  of 
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incoming  updates  from  these  force  areas,  and  thus  is  functioning  at  a  high  level  during 
periods  of  action  such  as  these.  The  Avn  Reg  HQ  shows  a  very  high  level  of  inform  others  of 
action  /  decisions,  which  follows  from  the  HQs  function  of  coordinating  and  following  the 
actions  of  the  helicopter  contingent  in  the  blue  force.  That  is,  information  received  from 
the  helos  themselves  is  digested  and  disseminated  to  the  JOSCC  staff  as  the  battle 
progresses. 


Table  7:  Critical  events  during  each  time  interval  in  mission  3 


l  Brief  status 

i  Inform  others  of  actions/decisions 


Discuss  actions/decisions 


Figures  10:  Levels  of  behaviour  relating  to  status  and  actions  in  the  Bde  HQ  during  mission  3 
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Figure  11:  Levels  of  behaviour  relating  to  status  and  actions  in  the  JOSCC  during  mission  3 


Figures  12:  Levels  of  behaviour  relating  to  status  and  actions  in  the  Avn  Reg  HQ  during  mission  3 

Figures  13  to  16  show  the  levels  of  information-related  behaviour  occurring  during 
mission  4,  while  figures  17  to  20  show  the  levels  of  behaviour  relating  to  the  commander  s 
perception  of  the  situation  and  necessary  actions  during  this  mission.  These  figures  should 
be  viewed  in  conjunction  with  Table  8  for  the  list  of  critical  events.  It  can  be  seen  that 
initially,  the  Bde  HQ  has  high  levels  of  receive  information  (during  intervals  1  8),  which 

becomes  high  levels  of  request  information  during  intervals  9  and  11.  These  are 
accompanied  by  very  high  levels  of  the  behaviours  indicate  information  on  map  send 
information  and  confirm  information.  The  JOSCC  shows  its  highest  levels  in  the  indicate 
information  on  map  and  send  information  categories,  with  additional  high  levels  observed  in 
the  request  and  confirm  information  categories.  The  Avn  Reg  HQ  displays  high  leve  s  o 
send  and  receive  information  across  all  time  intervals,  with  particularly  high  levels  occurring 
at  intervals  1,  and  6  and  7  respectively.  Confirming  and  requesting  information  also  show 
relatively  high  levels  across  the  majority  of  time  intervals.  Locon  shows  more  moderate 
levels  of  behaviour  across  time  intervals  (that  is,  request  information,  send  information, 
indicate  information  on  map,  and  anticipate  information  needs),  with  the  exception  of  receive 
information,  which  occurs  at  a  high  level  throughout  and  peaks  during  intervals  5, 6, 13 

and  14. 
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Time  block 


B  Request  ■  Receive  □  Send  □  Indicate  on  ma| 

■  Explain  ■  Confirm  ■  Anticipate 


Figure  13:  Levels  of  information  handling  behaviour  in  the  Bde  HQ  during  mission  4 


Figure  14:  Levels  of  information  handling  behaviour  in  the  JOSCC  during  mission  4 


Figure  15:  Levels  of  information  handling  behaviour  in  the  Avn  Reg  HQ  during  mission  4 
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Figure  16:  Levels  of  information  handling  behaviour  in  the  Locon  during  mission  4 

It  is  interesting  to  note  that  the  most  frequent  occurrences  of  anticipatory  behaviour  were 
found  in  Locon.  This  supports  the  previous  assertion  that  anticipatory  behaviours  (ie. 
anticipate  information  needs  of  others,  in  this  instance)  will  tend  to  increase  in  frequency  as 
the  team  members  become  more  familiar  with  each  other,  as  Locon  was  populated  with 
Military  staff  local  to  DSTO  who  had  worked  together8  previously. 


Table  8:Critical  events  table  for  mission  4 


Time 

interval 

1 

2 

3 

4 

5 

6 

7 

. 

8 

9 

10 

11 

12 

13 

14 

15 

16 

Event 

Troop  en  BMPs  contacted 

Prepare  for  handover  to  FA18s 

Weevil  attempting  to  withdraw  &  break 
en  contact 

Falcon  sth  &  central  CSs  in  contact  with 
en  T-80s:  must  picket  &  wait  for  helo  spt 

Confusion  over  grid  squares  and  clear 
ground  for  Falcon 

Weevil  to  laser  designate  for  helos 

Falcon  hit  by  T-80s  &  needs  spt;  helos  id 
targets  in  TAI3,  request  permission  to 
engage 

Orders  to  Sabre  &  Mitre  re;  next  targets 
&  advances 

Avn  Reg  cmdr  orders  concentration  of 
timed  fire;  Arty  &  fast  air  splash  TAI2 

T-80s  in  minefield  and  fast  air  targets 
destroyed;  Obj  1  &  Manbulloo  secured 

Weevil  to  send  BDA  re:  BTR80s  &  en  C2 
node  status 

En  vehicles  detected  withdrawing  to  NW 
of  river;  Knots  Crossing  secured;  no 
comms  to  fast  air 

Detect  hostile  jets,  air  raid  warning; 

BTR80  Trp  contacted 

En  surrender  accepted 

8  Including  training  together  at  PT  and  participating  in  the  social  activities  of  the  DSTO  Division  to  which 
they  were  assigned. 
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Figure  17:  Levels  of  status  and  action-related  behaviour  in  the  Bde  HQ  during  mission  4 


In  terms  of  the  Status/ Action  category  of  behaviours  (ie.  relating  to  the  commander's 
perception  of  the  situation),  the  Bde  HQ  (Figure  18)  shows  a  mixture  of  behaviour  levels 
across  time  intervals,  with  high  levels  of  all  three  behaviours  overall,  and  peaks  at 
different  times.  Brief  status  peaks  during  intervals  5  and  16,  while  inform  others  of  actions/ 
decisions  peaks  during  intervals  9  and  12,  and  discuss  actions/decisions  peaks  during 
intervals  4, 7, 10  and  11.  The  JOSCC,  by  contrast,  shows  very  high  levels  of  discuss  actions/ 
decisions  throughout  the  mission,  accompanied  by  low  levels  of  brief  status  (Figure  19).  A  vn 
Reg  HQ  (Figure  20)  shows  high  levels  of  inform  others  of  actions  /  decisions  and  discuss 
actions  /  decisions  during  intervals  2,  6  and  12.  Peaks  in  these  behaviours  occur  during 
intervals  13  and  14,  and  3, 10  and  15  respectively.  The  brief  status  behaviour  only  occurs 
during  interval  9,  but  does  occur  at  a  very  high  level  at  this  time.  Locon  (Figure  21),  by 
contrast  with  the  other  locations,  shows  a  moderate  level  of  all  three  behaviours  across  all 
time  intervals.9 


9  This  possibly  reflects  the  fact  that  the  Locon  staff  were  simulating  the  roles  of  multiple  Battle 
Groups  (BGs)  and  Combat  Teams  (CTs)  throughout  the  missions. 
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Figure  19:  Levels  of  status  and  action-related  behaviour  in  the  Avn  Reg  HQ  during  mission  4 


Figure  20:  Levels  of  status  and  action-related  behaviour  in  Locon  during  mission  4 


It  appears  that  there  is  a  general  trend  towards  the  prevalence  of  particular  behaviours 
during  (or  immediately  prior  to  or  following)  events  occurring  during  the  battle.  The 
trends  were  expected  to  be  stronger  given  that  the  behaviours  observed  related  to  the 
handling  and  treatment  of  information  -  which  is  at  a  premium  during  battle  and 
particularly  during  high  tempo  operations. 

Unfortunately,  there  are  no  clear,  strong  and  consistent  patterns  emerging  from  this  study 
which  can  be  said  to  inform  researchers  of  the  possible  effects  of  new  technology  on  the 
performance  of  the  subject  team.  This  lack  may  relate  to  several  causes,  including: 

•  The  artificiality  of  the  situation 

•  New  and  unfamiliar  tools  being  provided  to  the  subjects,  with  time  constraints  on 
training  and  familiarisation 

•  The  disruptions  caused  by  system  breakdowns  in  the  wargame 

•  The  unfamiliarity  of  the  subjects  with  their  team  mates 

In  addition,  it  may  be  that  the  behavioural  measures,  while  suitable  for  use  in  the  field 
environment  during  exercises  or  actual  operations,  may  not  be  suited  to  the  laboratory 
environment  in  their  current  form. 
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3.2.1  Correlations  between  team  behaviours 

Table  9  shows  the  significant  correlations  between  behaviours  in  each  location  across 
missions  during  Exercise  Prowling  Pegasus.  There  are  several  highly  significant 
correlations  evident  in  the  table,  and  correct  interpretation  of  these  may  require  returning 
to  the  raw  data  to  clarify  the  relationships  elicited  from  within  the  data.  For  example,  the 
very  high  significant  negative  correlation  (r  =  -0.838)  between  critical  event  and  receiving 
information  in  Locon  (bold  in  Table  9)  is  not  intuitive  until  the  raw  data  is  examined.  It  can 
be  seen  then  that  this  negative  relationship  simply  indicates  that  there  is  a  time  lag 
between  the  critical  events  occurring  and  the  receipt  of  information  by  Locon,  which 
means  that  the  two  zero  scores  for  the  critical  events  correlate  with  the  highest  levels  of 
receipt  of  information. 

Moderately  high  correlations  between  critical  events  and  the  requesting  and  sending  of 
information,  as  well  as  the  informing  others  of  actions  and  decisions  taken  at  the  Bde  HQ  level 
can  be  expected,  as  there  is  an  obvious  need  for  the  passage  of  troop  information  to  and 
from  the  command  level.  The  commander's  SA  must  be  maintained  and  updated  as  the 
situation  changes  with  the  progress  of  battle. 

There  are  interesting  trends  apparent  in  this  Bde  group,  however.  It  is  clear  that  there  was 
an  increasing  tendency  across  missions  for  Bde  HQ  staff  to  indicate  information  on  the 
battlemap  (located  on  the  large  Smartboard  screen).  In  addition,  the  JOSCC  staff  showed 
an  increasing  tendency  to  send  information  to  other  locations  as  well  as  to  discuss  information 
with  other  staff  in  their  location,  while  the  level  of  explain  information  behaviour  showed  a 
strong  tendency  to  decrease  over  missions.  This  is  possibly  due  to  the  increased  familiarity 
of  the  staff  with  the  Smartboard  technology,  therefore  leading  to  their  increased  ability  to 
pinpoint  desired  information  of  their  own  accord. 


Table  9:  Pearson  correlation  coefficients  for  behaviours  across  missions  2, 3  and  4  in  each  location 


Behaviour  1 

Behaviour  2 

Pearson  Correlation 
Coefficient  (r) 

Probability 

_ M _ 

Bde  HQ 

Critical  event10 

Request  information 

0.315 

P  <  0.05 

Send  information 

0.316 

P  <  0.05 

Inform  others  of  actions 

-0.434 

P  <  0.01 

Request  information 

Indicate  information  on  map 

0.316 

P  <  0.05 

Brief  status 

-0.306 

P  <  0.05 

Mission  number 

Indicate  information  on  map 

0.406 

P  <  0.01 

JOSCC 

Mission  number 

Send  information 

0.591 

P  <  0.01 

Explain  information 

-0.752 

P  <  0.01 

Discuss  actions /decisions 

0.657 

P  <  0.01 

10  NB:  ‘Critical  event’  is  not  a  behaviour,  rather  it  is  a  variable  in  the  database  indicating  that  an  event  or 
events  took  place  during  a  given  time  interval. 
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Behaviour  1 

Behaviour  2 

Pearson  Correlation 
Coefficient  (r) 

Probability 

_ M _ 

Request  information 

Send  information 

-0.316 

P  <  0.05 

Receive  information 

Brief  status 

0.351 

P  <  0.05 

Inform  others  of  actions 

0.424 

P  <  0.01 

Send  information 

Explain  information 

-0.400 

P  <  0.01 

Confirm  information 

-0.349 

P  <  0.05 

Brief  status 

0.314 

P  <  0.05 

Discuss  actions /decisions 

0.435 

P  <  0.01 

Explain  information 

Discuss  actions /decisions 

-0.648 

P  <  0.01 

Brief  status 

Discuss  actions /decisions 

0.328 

P  <  0.05 

Inform  others  of  actions 

0.808 

P  <  0.01 

Avn  Reg  HQ 

Request  information 

Receive  information 

0.419 

P  <  0.01 

Confirm  information 

0.308 

P  <  0.05 

Brief  status 

0.352 

P  <  0.05 

Receive  information 

Send  information 

0.473 

P  <  0.01 

Brief  status 

0.569 

P  <  0.05 

Send  information 

Inform  others  of  actions 

0.420 

P  <  0.01 

Discuss  actions /decisions 

0.618 

P  <  0.05 

Explain  information 

Brief  status 

0.469 

P  <  0.01 

Discuss  actions 

Inform  others 

0.672 

P  <  0.01 

Critical  event 

Receive  information 

-0.838 

P  <  0.01 

Receive  information 

Discuss  actions /decisions 

0.647 

P  <  0.01 

Send  information 

Indicate  information 

0.628 

P  <  0.01 

Inform  others  of  actions 

0.682 

P  <  0.01 

Indicate  information  on  map 

Explain  information 

0.561 

P  <  0.05 

Confirm  information 

0.581 

P  <  0.05 

Inform  others  of  actions 

0.732 

P  <  0.01 

Explain  information 

Confirm  information 

0.611 

P  <  0.05 

Anticipate  information  needs 

0.518 

P  <  0.05 

Inform  others  of  actions 

0.616 

P  <  0.05 

Brief  status 

Discuss  actions /decisions 

0.667 

P  <  0.01 

3.2.2  Correlations  between  information  handling  behaviour  and  a  basic 
performance  measure 

The  basic  performance  measure  used  in  this  report  is  the  kill/loss  ratio,  which  gives  an 
indication  of  the  level  of  concomitant  red  and  blue  losses.  The  total  losses  to  both  sides 
over  time  are  illustrated  in  Figures  21  and  22. 
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Figure  22:  Red  kills  and  blue  losses  during  mission  2 


Figure  23:  Reds  kills  and  blue  losses  during  missions  3  (left)  and  4  (right) 

The  correlations  between  the  behaviours  observed  in  the  staff  at  the  various  locations  and 
the  number  of  red  kills  and  blue  losses  are  shown  in  Tables  10a  and  10b.  It  is  apparent  that 
there  are  few  clear  relations  between  these  variables,  with  the  exception  of  moderate 
positive  relationships  between  information  traffic  in  all  locations  (requesting  and  receiving 
information)  during  missions  2  and  3.  The  Avn  Reg  HQ  results  in  Table  10b  indicate  that 
the  strongest  tendencies  for  discussion,  receipt  and  briefing  of  information  lay  with  the 
staff  in  this  location,  possibly  because  they  were  isolated  from  the  Bde  HQ  and  JOSCC 
tents,  which  were  located  adjacent  to  one  another  in  the  atrium  and  therefore  allowed 
more  simultaneous  communal  observation  of  information  by  staff.  In  addition,  the  Avn 
Reg  HQ's  role  of  communicating  with  the  ARH  pilots  and  tasking  the  helos  would  mean 
that  they  had  a  more  direct  role  to  play  in  this  type  of  information  handling  when  critical 
events  occurred  during  missions. 

Overall,  then,  the  correlations  between  behaviour  and  the  numbers  of  red  kills  and  blue 
losses  illustrated  in  Table  10  simply  support  the  notion  that  information  is  the  most 
important  commodity  in  the  headquarters  locations.  It  remains  for  future  work  — 
preferably  in  a  field  setting  —  to  investigate  whether  there  are  in  fact  consistent  observable 
relationships  between  these  behavioural  factors  and  the  performance  outcomes  of  the 
battles  fought. 
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Table  10:  Significant  correlations  betiveen  behaviours/events  and  numbers  of  red  kills/blue  losses 


a)  During  each  mission  (across  all  locations) 


Item  1 

Behaviour 

Pearson  Correlation 
Coefficient  (r) 

Probability  ( p ) 

Mission  2 

Reds  destroyed 

Request  information 

0.217 

P  <  0.05 

Mission  3 

Blue  Losses 

Request  information 

0.264 

P  <  0.05 

Reds  destroyed 

Request  information 

0.396 

P  <  0.05 

Reds  destroyed 

Receive  information 

0.347 

P  <  0.05 

b)  In  each  location  (across  all  missions) 


Item  1 

Behaviour 

Pearson  Correlation 
Coefficient  (r) 

Probability  (p) 

Avn  Reg  HQ 

Reds  destroyed 

Discuss  actions/  decisions 

0.302 

P  <  0.05 

Reds  destroyed 

Receive  information 

0.576 

P  <  0.05 

Brief  status 

0.667 

P  <  0.01 

3.3  Tool  use 

Tool  use  in  each  of  the  different  locations  is  shown  in  Figures  24  to  26.  Mission  codes 
relevant  to  the  figures  below  are  stated  in  Table  11.  It  can  be  seen  that  the  use  of  SA  tools 
and  radios  occurs  at  a  relatively  high  level  across  missions  for  the  Bde  HQ,  the  JOSCC,  and 
the  Avn  Reg  HQ  (shown  in  Figures  24,  25  and  26  respectively).  The  highest  reliance 
appears  to  be  on  the  use  of  the  SA  tool  (in  this  instance,  the  Landscape  software)  in  the 
JOSCC  and  Avn  Reg  HQ,  with  the  Bde  HQ  showing  similar  levels  of  use  for  both  the  SA 
tool  and  the  radios.  SA  tool  use  dropped  away  quite  sharply  during  mission  4  in  the  Avn 
Reg  HQ,  and  this  was  later  found  to  be  a  product  of  the  low  level  of  staffing  in  that 
location’1  rather  than  any  preference  for  other  modes  of  information  handling. 

Table  11:  Mission  codes  used  in  Figures  24  to  29 


Codes 

Mission  Numbers 

1 

2 

2 

3 

3 

4 

11  Pers  comm.  CW4  R  Huffman. 
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Figure  24:  Tool  use  and  kill  floss  ratios  in  the  Bde  HQ 

It  is  apparent  that  there  is  a  strong  correlational  relationship  between  the  use  of  certain 
tools  (such  as  the  SA  tool  and  radio)  and  the  kill  loss  ratio  shown  in  Figures  24  to  26.  The 
values  of  these  correlations  are  shown  in  table  12.  Significant  moderate  positive 
relationships  between  the  performance  measure  (ie.  the  kill/ loss  ratio)  and  the  use  of  the 
SA  tool,  paper  maps  and  radios  are  seen  in  the  Bde  HQ,  indicating  that  improvements  in 
performance  are  associated  with  higher  levels  of  use  of  this  equipment  by  staff.  The 
JOSCC,  by  contrast,  shows  a  significant  negative  correlation  between  the  kill/loss  ratio 
and  the  use  of  the  SA  tool  and  Stealthviewer.  The  cause  for  this  is  unclear,  particularly 
given  the  results  obtained  for  the  Bde  HQ,  but  it  may  be  an  artefact  of  the  artificiality  of 
the  experimental  command  location  setups. 

The  Avn  Reg  HQ  shows  highly  significant,  strong  negative  correlations  between  the 
kill/loss  ratio  performance  measure  and  battlemap  and  Stealthviewer  use,  indicating  that 
improved  performance  was  associated  with  lower  levels  of  use  of  these  tools.  Radio 
communications  at  this  location,  on  the  other  hand,  showed  a  significant  strong  positive 
relationship  with  performance,  indicating  that  there  is  still  a  strong  reliance  of  the  pilots 
and  those  in  base  command  posts  on  voice  communications  for  the  updating  of  individual 
(and  group)  SA. 


Figure  25:  Tool  use  and  kill/loss  ratios  in  the  JOSCC 
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Tool  use  compared  with  kill/loss  ratios  for  Avn  Reg  HQ 
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Figure  26:  Tool  use  and  kill/loss  ratios  in  the  Avn  Reg  HQ 


Table  12:  Correlation  coefficients  for  the  relationships  between  tool  use  and  kill/loss  ratios  in  each 
location  across  missions 


Item  1 

Item  2 

Pearson's  r  Correlation 
Coefficient 

Probability 

<P> _ 

Bde  HQ 

Kill/Loss  ratio 

SA  tool  use 

0.486 

P  <  0.05 

Radio  use 

0.583 

P  <  0.05 

Paper  map  use 

0.583 

P  <  0.05 

fOSCC 

Kill /Loss  ratio 

SA  tool  use 

-0.372 

P  <  0.05 

Radio  use 

0.911 

P  <  0.01 

Paper  map  use 

0.812 

P  <  0.01 

Stealthviewer  use 

-0.912 

P  <  0.01 

Avn  Reg  HQ 

Kill/ Loss  ratio 

Radio  use 

0.911 

P  <  0.01 

Battlemap  use 

-0.910 

P  <  0.01 

Stealthviewer  use 

-0.911 

P  <  0.01 

3.3.1  Tool  use  and  team  behaviour 

Figures  27  to  29  illustrate  the  significant  relationships  between  behaviours  and  tool  use 
that  are  listed  in  Table  13  below.  Clearly,  the  SA  tool  in  the  Bde  HQ  was  of  use  to  staff 
when  briefs  of  events,  or  asset  status  or  locations  were  occurring.  It  is  also  apparent  that 
the  Stealthviewer  (images  fed  to  the  Bde  HQ  from  the  TUAV)  was  sought  after  for 
confirmation  of  reported  information,  and  that  the  MoDSAF  screen  was  used  as  a  back  up 
for  the  SA  tool  in  terms  of  asset  information. 
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Figure  27:  Significant  correlational  relationships  between  tool  use  and  behaviour  in  the  Bde  HQ 


JOSCC:  relationships  between  tool  use,  behaviour  and  the  kill/loss 
ratio  during  each  mission 
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Figure  28:  Significant  correlational  relationships  between  tool  use  and  behaviour  in  the  JOSCC 


Figure  28  shows  that  in  the  JOSCC,  the  battlemap  was  often  used  concomitant  with  the 
Stealthviewer  during  mission  2,  and  its  use  was  also  commonly  associated  with 
behaviours  such  as  receiving  information,  brief  status,  and  inform  others  of  actions/decisions. 
The  use  of  MoDSAF  was  frequently  observed  concomitant  with  sending  information, 
indicating  that  staff  tended  to  check  or  obtain  information  from  the  MoDSAF  screen  prior 
to  sharing  this  information  with  staff  in  other  locations. 

Figure  29  shows  that  the  use  of  the  SA  tool  was  often  backed  up  with  the  use  of  paper 
maps  in  the  Avn  Reg  HQ.  This  appears  to  indicate  that  the  staff  are  still  unsure  in  the  use 
of  the  SA  tool  (due  to  minimal  training  and  lack  of  experience  with  the  tool  at  this  stage)  or 
that  (as  previously  stated  in  the  case  of  the  Avn  Reg  HQ)  the  lack  of  staff  numbers  — 
possibly  combined  with  the  lack  of  experience  —  make  the  use  of  this  tool  inefficient  and 
relatively  unreliable  compared  with  the  well  known  pen  and  paper  method.  Use  of  the 
battlemap  and  paper  maps  are  often  concomitant  with  the  use  of  the  radio,  however 
behaviours  such  as  explaining  information  and  brief  status  showed  a  negative  correlation 
with  radio  use,  indicating  that  the  tendency  is  to  deal  with  staff  in  situ  and  that  the  radio 
was  therefore  not  needed.  The  negative  relationships  between  these  two  behaviours  -  in 
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addition  to  the  use  of  both  battlemap  and  paper  maps  -  indicates  that  the  tendency  is  for 
the  staff  member  carrying  out  the  briefing  or  explanation  to  acquire  the  necessary  pieces  of 
information  prior  to  beginning  the  brief /explanation,  and  then  to  carry  these  out  without 
the  use  of  prompts. 


Table  13:  Significant  correlations  between  tool  use  and  team  behaviour  across  missions  in  each 
location 


Tool  Used 

Behaviour/Tool  used 

Pearson 
Correlation 
Coefficient  (r) 

Probability  (p) 

BdeHQ 

SA  tool 

Brief  status 

0.346 

P  <  0.05 

Stealthviewer 

Confirm  information 

0.308 

P  <  0.05 

ModSAF 

Indicate  information  on  map 

0.395 

P  <  0.01 

JOSCC 

Battlemap 

Stealthviewer 

0.550 

P  <  0.01 

Receive  information 

0.353 

P  <  0.05 

Brief  status 

0.600 

P  <  0.01 

Inform  others  of  actions 

0.846 

P  <  0.01 

ModSAF 

Send  information 

0.465 

P  <  0.01 

Avn  Re^  HQ 

SA  tool 

Paper  map 

0.354 

P  <  0.05 

Radio 

Battlemap 

0.427 

P  <  0.01 

Paper  map 

0.500 

P  <  0.01 

Explain  information 

-0.347 

P  <  0.05 

Brief  status 

-0.305 

P  <  0.05 

Battlemap 

Brief  status 

-0.335 

P  <  0.05 

Paper  map 

Explain  information 

-0.303 

P  <  0.05 

Brief  status 

-0.457 

P  <  0.01 

Locott 

Paper  map 

Explain  information 

0.681 

P  <  0.01 

Confirm  information 

0.507 

P  <  0.05 

Anticipate  information  needs 

0.717 

P  <  0.01 

ModSAF 

Explain  information 

0.691 

P  <  0.01 

Confirm  information 

0.808 

P  <  0.01 

Paper  map 

0.683 

P  <  0.01 
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Figure  29:  Significant  correlational  relationships  between  tool  use  and  behaviour  in  the  Avn  Reg 
HQ 

Locon  is  not  represented  graphically,  as  observations  were  only  carried  out  on  this 
location  during  mission  4,  and  therefore  no  pattern  can  be  illustrated  across  missions. 
Clearly,  though,  there  are  very  strong  positive  relationships  between  the  use  of  the  paper 
maps  and  both  explain  information  and  confirm  information  behaviours,  as  well  as 
anticipating  information  needs.  The  use  of  MoDSAF  also  correlates  very  strongly  with 
explaining  information  and  confirming  information,  as  well  as  the  use  of  the  paper  map.  The 
population  of  Locon  with  the  LOD  military  (ie.  local  DSTO-assigned  military  staff)  may 
have  contributed  to  the  higher  incidence  of  anticipatory  behaviour  recorded  here,  as  these 
staff  have  had  more  prior  experience  working  with  each  other  than  the  majority  of  the 
military  staff  brought  in  from  the  various  base  locations.  In  addition,  they  are  more 
familiar  with  the  workings  of  MoDSAF  than  'outsiders',  and  have  developed  a  level  of 
familiarity  with  the  functions  provided  by  these  technologies. 

3.4  Performance  measures  and  perceptions  of  achieved  intent 

During  the  AARs  held  following  missions  2,  3  and  4,  staff  from  the  different  locations 
were  asked  to  outline  several  important  items  of  information  and  comment  on  the 
effectiveness  of  their  mission(s)  and  the  achievement  of  intent.  These  areas  included: 

•  The  mission 

•  The  task(s) 

•  The  endstate  achieved 

•  The  tactics  used 

•  The  weapons  and/ or  sensors  used 

•  The  effectiveness  of  the  mission 

•  The  achievement  of  intent 

Summarised  versions  of  the  AAR  outcomes  can  be  found  in  Appendix  C. 

The  overarching  intent  for  all  missions  followed  the  guideline: 

'ITF  is  to  capture  Katherine  in  order  to  dislodge  MAF  elements  and  force  them  to  the 
North  West  in  order  to  set  the  preconditions  for  their  destruction  in  detail.' 
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This  was  the  guidance  that  determined  the  planning  conducted  by  the  sub-unit 
commanders,  and  the  methods  to  be  used  for  the  operational  phases  of  the  missions.  The 
crucial  element  of  the  mission  outlined  above  is  that  the  enemy  was  to  be  dislodged  from 
within  Katherine  and  destroyed  in  detail  in  the  open  areas  to  the  Northwest.  One  of  the 
intentions  here  was  to  minimise  the  civilian  casualties  in  the  town  itself.  That  is,  it  was  not 
adequate  to  simply  destroy  the  enemy,  it  was  intended  for  this  to  occur  in  a  carefully 
controlled  area.  Once  this  type  of  issue  is  considered,  it  becomes  abundantly  clear  that  a 
single  simple  measure  of  effectiveness  (such  as  the  kill/loss  ratio  used  earlier,  for  example) 
is  not  adequate  to  judge  the  quality  of  the  outcomes  achieved.  There  should  be  a 
combination  of  measures  used  which  take  into  account  all  the  important  aspects  of  the 
battle. 

Thus,  the  perception  of  achieved  intent  can  work  interestingly  with  the  actual  outcomes  in 
terms  of  the  number  of  red  kills  and  blue  losses,  and/or  the  kill/ loss  ratio  examined  in 
conjunction  with  the  location  of  the  enemy  assets  that  were  destroyed. 


Figure  30:  Reds  surviving  and  reds  destroyed  (and  where)  during  mission  2 

A  figure  such  as  30  breaks  down  the  actual  destruction  of  red  assets  and  the  locations  of 
the  red  kills,  which  can  give  a  clearer  indication  of  the  actual  achievement  of  intent  in  its 
most  precise  form.  It  can  be  seen  that  the  number  of  reds  destroyed  in  Katherine  is  initially 
relatively  low,  and  that  those  destroyed  elsewhere  is  quite  high.  The  number  actually 
destroyed  within  the  designated  zone  is  very  low.  Comparing  this  with  Figures  31  and  32 
(below),  it  is  clear  that  only  at  the  end  of  mission  4  is  the  intent  to  destroy  the  enemy  in  the 
area  North  West  of  Katherine  met  to  any  great  degree.  The  number  of  reds  destroyed  in 
this  area  rises  sharply  during  the  third-last  time  block,  however  there  is  still  a  high  level  of 
red  destruction  occurring  in  the  'elsewhere'  category  and  also  within  Katherine  itself. 
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Figure  31:  Reds  surviving  and  reds  destroyed  (and  where)  during  mission  3 


Figure  32:  Reds  surviving  and  reds  destroyed  (and  where)  during  mission  4 

Although  there  should  be  a  relatively  clear  relationship  between  the  outcomes  of  the 
wargames  and  the  perception  of  achieved  intent,  more  work  needs  to  be  conducted  in 
terms  of  relating  the  measures  of  performance  chosen  to  the  outcomes  of  the  wargames 
themselves  to  elucidate  this  relationship.  As  can  be  seen  from  the  figures  above  and  the 
comment  available  in  the  AAR  summary  in  Appendix  C,  perceptions  of  successfully 
achieved  missions  and  the  meeting  of  intent  incorporate  factors  other  than  simple 
comparisons  between  what  happened  and  the  way  the  original  commander's  intent  was 
stated.  In  order  to  successfully  make  statements  regarding  these  relationships,  an 
understanding  has  to  be  sought  from  the  military  staff  involved.  This  in  itself  may  not  be 
simple,  in  that  different  commanders  tend  to  have  individual  styles  of  command  and 
possibly  perceptions  of  what  constitutes  acceptable  adherence  to  the  commander's  intent 
in  conjunction  with  the  achievement  of  aims.  This  will  be  addressed  in  future  work. 


35 


DSTO-TR-1521 


3.5  Higher  level  measures  and  behaviour 

In  terms  of  the  system  of  systems  analysis  desired  by  researchers  at  DSTO,  the  outcomes  of 
the  behavioural  observation  have  not  proved  to  be  good  indicators  of  the  outcomes 
produced  in  the  artificial  laboratory  environment.  A  list  of  possible  causes  for  this  is 
outlined  in  section  3.2.  It  is  intended,  however,  that  future  work  will  attempt  to  delineate  a 
relationship  between  performance  measures  at  all  levels,  wargame  outcomes,  and 
behavioural  indicators  in  a  variety  of  settings  -  including  field  exercises  and  (potentially) 
operational  settings. 

In  addition,  future  work  will  attempt  to  investigate  relations  between  the  higher  level 
outcome  measures  and  the  measures  of  behaviour  utilised  here.  There  are  several  types  of 
higher  level  measures  suggested  for  use  in  this  type  of  experiment.  They  are  outlined  in 
brief  below,  and  follow  a  list  of  potential  measures  outlined  during  the  planning  process 
for  the  Prowling  Pegasus  experiment. 

Measures  of  Force  Effectiveness  (MOFEs) 

These  are  a  series  of  measures  designed  to  elucidate  the  level  of  effectiveness  of  the  force 
under  certain  conditions.  They  include: 

MOFE1:  the  total  time  taken  to  expel  all  enemy  from  Katherine 

MOFE2:  a  time  integral  of  the  enemy's  capability  in  town  (which  requires  a  graph  of  the 
enemy  capability  in  the  town  over  time).  Enemy  capability  can  be  taken  as  equivalent  to 
the  number  of  enemy  to  a  first  approximation. 

MOFE3:  the  number  of  enemy  destroyed  outside  the  town.  Aspects  of  this  measure 
include  (a)  the  number  of  enemy  escaping,  (b)  the  number  of  enemy  destroyed  inside  the 
town. 

Measures  of  Force  Cost  (MOFCs) 

MOFC1:  the  overall  cost  of  the  force  in  terms  of  assets  and  equipment  (a  list  of  numbers 
and  types  of  assets  is  necessary  for  this.  Eg.  500  infantry,  40  SF,  30  LAVs,  etc) 

MOFC2:  the  number  of  losses  to  the  blue  forces  (numbers  and  types  of  assets,  as  well  as 
troops) 

MOFC3:  the  number  of  civilian  losses 

Measures  of  Effectiveness  (MOEs) 

MOE1:  includes  items  such  as 

•  The  number  of  targets  destroyed  by  the  LAS 

•  The  number  of  targets  assigned  to  the  LAS  but  not  destroyed  by  it 

MOE2:  includes  items  such  as 

•  The  rate  of  targeting  (ie.  a  graph  of  the  numbers  targeted  for  and  by  the  LAS  as  a 
function  of  time) 

•  The  number  of  elements  of  the  LAS  remaining  and  untargeted 
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Measures  of  Cost  (MOCs) 

MOC1:  includes  such  items  as 

•  The  capital  value  of  the  LAS  (a  list  of  numbers  and  types  of  assets  and  troops 
within  the  LAS) 

•  LAS  losses 

•  Fratricide  due  to  the  LAS  (both  own  force  and  civilian) 

•  Collateral  damage  caused  by  the  LAS 

MOC2:  includes  such  items  as 

•  Vulnerability 

•  The  total  time  elements  are  within  the  range  of  enemy  weapons 
Measures  of  Performance  (MOPs) 

These  are  measures  of  the  times  taken  to  complete  the  phases  of  the  procedures 
implemented  and  hence,  they  will  change  if  different  procedures  are  used.  These  should 
be  converted  to  rates  (ie.  the  inverse  of  time) 

MOP1: 

Tasking  time  (this  includes  brigade  level  planning,  preparation  of  orders  to  the  BG, 
communication  and  receipt  of  intent  and  orders).  It  is  the  start  time  to  the  finish  time 
(when  the  orders  reached  the  Regimental  HQ). 

Mission  planning  time.  This  is  from  the  receipt  of  orders  to  the  actuality  of  having  the  plan 
in  the  helicopter  and  other  LAS  elements.  This  can  be  further  broken  down  into  planning 
times  for  each  element  of  the  BG. 

Time  to  target  area.  This  is  from  the  time  the  plan  is  received  to  actually  reaching  the  NAI. 
This  should  involve  separate  measures  for  each  of  the  elements  involved. 

Search  time  for  the  LAS  element  doing  the  target  acquisition.  This  is  from  the  time  of  reaching 
the  NAI  to  the  time  of  target  location. 

Engagement  time.  The  time  from  locating  the  target  to  weapon  engagement,  including  the 
handover  time  for  third  party  targeting. 

Retasking  time.  The  time  from  weapon  engagement  on  one  target  to  the  receipt  of  retasking 
orders. 

Costs  at  this  level  include: 

•  The  ratio  of  numbers  of  threats  known  to  the  total  number  of  threats 

•  The  acceptable  risk  factor  (in  the  experimental  situation,  this  would  be  a  subjective 
assessment  by  the  SME,  rated  on  a  scale  of  one  to  ten). 
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MOP2: 

The  contribution  of  various  system  components  to  the  MOPls  needs  to  be  measured,  and 
then  system  decomposition  and  influences  are  required.  A  series  of  comparative  baseline 
experiments  are  necessary  so  that  effects  of  system  components  can  be  assessed,  and  the 
MOPls  should  be  used  here  to  assess  the  impact  of  these  system  component  aspects.  The 
system  synergism  issues  should  be  taken  into  account,  which  means  that  the  true  impact 
of  the  technological  enhancements  can  only  be  made  apparent  if  procedures  designed  to 
harness  their  potential  are  implemented  alongside  the  system  components  themselves. 

It  is  suggested  that  the  measures  can  either  measure  the  performance  of  the  tools  in  terms 
of  their  provision  is  SA  to  the  subjects,  or  assume  that  SA  is  directly  related  to  the  ability  to 
perform  tasks  and  therefore  measure  task  performance. 

Tasking  time.  This  is  influenced  by  the  time  taken  to  acquire  the  SA  which  leads  to  the 
decision  to  task  the  LAS,  and  the  LSAS  should  have  an  impact  here.  It  should  also  impact 
on  other  contributing  aspects,  including  the  Brigade's  planning  process,  the  issuing  of 
orders,  and  the  receipt  of  intent. 

Mission  planning.  The  LSAS  will  impact  on  the  acquisition  of  the  SA  (including  red,  blue, 
and  the  environment  in  general)  and  the  understanding  of  their  interactions.  In  addition, 
the  mechanics  of  plan  production  and  communication  will  also  be  impacted. 

Time  to  target  area.  This  will  depend  on  aspects  of  platform  mobility,  but  also  the  SA 
related  to  the  platforms'  navigation  and  threat  avoidance. 

Search  time  to  target  acquisition.  This  will  depend  on  sensor  capabilities  and  surveillance 
procedures  (such  as  search  patterns  and  cooperative  surveillance),  and  should  be 
impacted  by  the  LSAS. 

Engagement  time.  The  LSAS  can  assist  the  handover  from  target  acquisition  to  the  weapon 
or  response  system. 

Retasking  time.  The  LSAS  can  provide  the  ability  for  collaborative  tasking,  and  so  can  assist 
retasking  'on  the  move'. 

Costs.  The  LSAS  can  reduce  the  vulnerability  of  blue  assets  by  providing  displays  of 
known  and  potential  threats,  including  their  engagement  ranges.  Mission  planning  can  be 
conducted  using  the  LSAS  to  utilise  terrain  features  in  order  to  avoid  these  threats. 

The  list  of  higher  level  performance  and  cost  measures  above  constitute  a  preliminary 
attempt  to  define  effective  and  meaningful  measures  which  will  allow  researchers  to  give 
some  significant  feedback  to  the  military  participants  as  well  as  supplying  valuable 
insights  into  the  workings  of  the  military  and  their  interaction  with  technology  in  a  variety 
of  situations. 
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4.  Conclusions 

Overall  then,  when  the  results  are  examined,  some  patterns  and  general  observations  can 
be  made. 

The  observed  ties  between  behaviours  and  events  in  the  different  locations  were  different, 
however  this  may  not  be  especially  meaningful  in  light  of  the  fact  that  subjects  in  each 
location  differed  on  several  dimensions.  That  is,  each  location  had  different  numbers  of 
subjects,  they  had  different  levels  of  experience  functioning  as  a  team  with  other  members, 
and  there  were  differing  levels  of  experience  in  their  assigned  roles. 

Overall  across  missions  2, 3  and  4,  the  subjects  tended  to  display  higher  levels  of  sending, 
requesting,  explaining  information ,  as  well  as  indicating  information  on  map  in  temporal 
proximity  to  the  occurrence  of  critical  events.  This  should  be  expected  as  the  team 
members  would  have  been  sharing  and  analysing  important  information,  both  for  its 
content  and  for  the  consequences  to  their  force.  In  the  Avn  Reg  HQ,  however,  this  was 
eclipsed  by  receiving  information  (although  sending  and  requesting  information  remained 
high)  as  they  tended  to  act  as  a  conduit  between  the  Bde  HQ  and  JOSCC,  and  the 
helicopters. 

In  keeping  with  expectations  as  the  increasing  levels  of  familiarity  and  experience  the 
participants  gained  during  the  exercise,  the  levels  of  anticipatory  behaviour  showed  a 
concomitance  in  all  locations. 

In  terms  of  observed  behaviour  and  its  correlations  with  the  basic  performance  measure 
(ie.  the  kill/loss  ratio)  across  the  locations,  there  was  little  in  the  way  of  an  overt  trend, 
since  the  correlations  were  relatively  weak  and  did  not  indicate  a  strong  relationship. 

Tool  use  levels  showed  that  there  was  a  difference  in  tool  preference  between  the 
locations,  with  the  Bde  HQ  consistently  preferring  radios  to  other  tools,  while  the  Avn  Reg 
HQ  and  JOSCC  both  clearly  preferred  the  SA  tool  during  missions  2  and  3,  reverting  to  a 
radio  preference  during  mission  4. 

The  preferential  use  of  particular  tools  for  specific  jobs  (ie.  the  SA  tool  for  the  briefings 
conducted  in  the  Bde  HQ)  were  noted,  and  the  popular  characteristics  of  the  tools  noted 
for  inclusion  in  future  design  specifications. 

The  achievement  of  intent  and  mission  success  measure  require  refinement  and 
adjustment,  as  there  are  no  clear  guidelines  at  this  stage  to  indicate  during  this  scenario 
what  constituted  acceptable  blue  losses,  and  what  level  of  enemy  destruction  was 
acceptable  within  the  Katherine  town  limits  (given  the  original  intent).  More  specific  data 
needs  to  be  sought  during  the  scientific  AARs  in  order  to  clarify  these  points  and  allow 
more  accurate  assessment  of  the  outcomes  of  the  experiment. 
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In  summary,  further  work  needs  to  be  conducted  on  the  relations  of  behaviour  and 
performance  in  these  settings.  As  the  relations  under  these  experimental  conditions  are 
relatively  tenuous,  more  rigid  controls  may  be  needed  in  order  to  strictly  test  the  outcomes 
of  adding  technological  enhancement  to  military  teams  performing  under  battle  stress.  It  is 
here  that  the  input  of  the  behaviour  systems /systems  perspective  approach  will  be 
valuable  in  determining  precise  points  of  error  and  the  origins  of  problematic  outcomes 
(which  can  occur  when  staff  are  unfamiliar  with  new  technologies).  These  issues  will  be 
addressed  in  future  studies  in  this  area. 
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Appendix  A:  Simple  behaviour  systems  models 
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Figure  Al:  Part  of  the  Feeding  behaviour  system  of  the  ratXFigure  3.14  from  Shettleworth) 
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Figure  A2:  A  section  of  the  Planning  behaviour  system  for  military  staff. 


42 


DSTO-TR-1521 


Appendix  C:  After  Action  Reviews:  summary  of 

results 

C.l.  AAR  1:  Missions  2  Tuesday  20/03/01 

ARH 

What  was  the  mission?  (Information  from  each  level) 

■  Route  for  the  helicopters 

■  Primary  task  -  clear  route  to  Manbuloo 

■  Security  for  force  troops  on  the  ground 
End  state 

Landing  Zone  (LZ)  at  Manbuloo  secured  to  continue  operations 
Tactics 

■  Both  to  clear  route  N  /NE 

■  Performed  normal  techniques  within  limits  of  sensors  and  visual  range 

Weapons/Sensors 

■  Combination  of  all  sensors 

■  Air  to  ground  missiles  (3000m  range) 

■  Cannon  for  ranges  less  than  this 

Effectiveness 

■  Covered  the  lack  of  reality 

■  Unrealistic  weapons  choice  (eg  hellfire  used  on  manpads) 

■  Not  as  effective  against  single  enemy  targets 

Was  the  Commanders  intent  covered?  Yes,  via  specific  orders. 

Feedback  from  ARH  to  Commander  (COMD)  helped  COMD  know  about  updates. 
Was  there  sufficient  freedom  of  action?  Yes.  Plenty  of  freedom  to  conduct  the  mission. 

SQNHQ  (NORTH) 

What  was  the  mission? 

■  Destroy  the  enemy  (en)  C2  node/s 

*  Preliminary  operation:  clear  NAI2  (GBAD)  in  conjunction  with  SF 

■  Phase  1:  destroy  C2  at  TAI  (West  of  Katherine) 

■  Locate  and  destroy  en  reserve(res) 

End  state 

GBAD,  NAI2  destroyed/neutralised 
En  C2  nodes /HQ  destroyed 
Res  destroyed 
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Tactics 

■  Maximum  standoff  range 

■  Ground  based  laser  designation  for  1st  two  targets 

■  Res  combination,  recon  and  standoff  range 

Weapons/Sensors 

■  Mainly  hellfire  and  ground  based  designation  (SF) 

Were  any  sensors  vital?  Yes,  optics. 

Effectiveness 

Achieved  end  states  and  was  effective. 

Why  destroy  GBAD?  CAIRS  not  available  and  destroying  GBAD  allowed  freedom  to  do 
the  required  tasks. 

SQN  HQ  (SOUTH) 

What  was  the  mission? 

■  Air  assault  security 

■  Secure  assault  elements 

■  Security  for  ground  forces 

Weapons/Sensors 

■  Hellfire  and  cannon 

■  UAV's  (couldn't  tell  where  the  target  was  from  it) 

Effectiveness 

■  fair 

Impact  of  Situation  Awareness  (SA)  tools 

Good.  Would  be  good  if  the  control  display  could  have  control  graphics  added  and  linked 
to  the  pilots  display,  (eg)  attack  by  fire  line  on  the  display. 

Control  Graphics  would  help  with  the  coordination  (eg)  Bn  HQ  to  Bde  could  have  plan 
with  continuos  graphics  and  linked  tot  eh  ARH  off  the  one  script  so  everyone  is  working 
to  the  same  plan. 

UAV  info  great,  but  the  link  wasn't  so  good.  Is  there  a  way  for  UAV  info  to  appear  in  ARH 
as  a  tool? 

With  respect  to  targets  on  SA  display  -  3  Tiers  of  target: 

■  Detection 

■  Recognition 

■  Identification 

IFF  (ID  Friend  or  FOE  systems  may  help  with  this) 

These  3  tiers  are  essential  for  SA.  Can  entities  be  identified  in  this  way  in  Battlescape? 
Once  en  is  confirmed,  it  should  be  entered  though  this  process  to  recognition  and  ID  and 
then  entered  into  the  system. 
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Pilot  doesn't  want  details  (ie  what,  where,  how  fast  etc)  -  wants  simple  symbol  on  screen 
for  en. 

One  Option  to  this . 

Pilot  wants  to  get  info  with  respect  to  where  contact  is,  doesn't  want  to  work  it  out  from 
grid  refs  etc.  Should  be  able  to  lase  via  link. 

A  menu  system  -  one  button-  to  enter  what  it  is  and  be  able  to  enter  quick  descriptor  is 
necessary,  and  then  lase  info  back  in  burst  or  via  link.  Detail  can  even  be  via  short  .wav 
burst.  Allows  them  to  get  on  with  job.  Can  do  active  or  passive  system  lasing. 

One  solution  of  many  -  minimum  disturbance  of  what  aircrew  is  doing. 

Comparisons  with  current  operational  helicopter  simulators  etc.  show  that  the  capability 
for  this  is  out  there.  (I  can  elaborate  on  this  if  you  like) 

UAVs 

What  was  the  mission? 

Locate  en  in  Manbuloo  and  the  res  force 

Tasks 

■  Locate  en  in  Manbuloo 

■  Prepare  to  lase  targets  for  ARH 

■  Find  res 

End  state 

Secure  airmobile  operation  and  identify  res 
Tactics 

■  Descend  through  to  area  move /ascend  to  ensure  out  of  weapons  range  then 
continue  to  next  position 

■  Had  to  unrealistically  descend  to  get  good  resolution 

■  Were  able  to  fly  by  Manbuloo  and  to  reserve 

■  If  still  able,  would  have  stayed  with  airmobile  area  until  retasked 

■  Lack  of  control  of  UAV.  Is  there  a  technical  glitch  here? 

■  Actual  plan  didn't  happen 

JOSCC  and  Bde  HQ  need  to  be  able  to  control  the  UAV  loc. 

UAV  is  a  good  cuing  device.  Preplanned  route  was  flown.  Operator  has  control  of  view 
not  the  flight  path  -  can  correct  this  with  another  ModSAF  box. 

BDE  HQ 

What  was  the  mission? 

■  Capture  Katherine  and  dislodge  en,  destroy  defence  and  en  reserve  to  N  of 
Katherine 
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Task 

■  Establish  strong  SF  and  Armd  recon  screen 

■  Secure  Manbuloo 

■  Destroy  GBAD 

■  Destroy  en  C2  and  res 

■  Attack  Katherine  -  dislodge  en  -attack  NW  Kath  and  destroy  en. 

End  state 

Successfully  capture  Katherine  with  minimum  damage  to  infrastructure.  Destroy/capture 
en  NW  of  Katherine. 

Tactics 

■  Establish  strong  SF/Armd  screen  early 

■  Conduct  preliminary  operation  -airmobile  to  Manbuloo 

■  Destruction  of  GBAD 

■  Destruction  of  C2/  res 

■  2  phase  attack 

■  phase  1-  Katherine 

■  phase  2  -  area  NW  of  Katherine 

Weapons/Sensors 

■  SF,  ARH,  Helos,  Armd  Recon,  UAV,  Mechanised  and  Motorised  Infantry,  guns, 
artillery 

Information  on  the  high  bridge  being  out  came  via  EW  data. 

Passed  control  of  assets  around  during  battle  to  ensure  tasks  completed  for  both  big  and 
local  pictures. 

Effectiveness 

■  Very  effective  battle  group 

■  Very  effective  ARH,  SF  and  Armd  Recon 

■  UAV  not  as  effective  as  hoped 

■  Control  of  UAV  -  JOSCC 

■  Was  almost  too  effective-  didn't  rout  en  in  Katherine  -instead  corralled  them  there. 

■  Perhaps  should  have  been  less  aggressive  to  west  of  river. 

■  Current  scenario  would  not  have  minimised  the  impact  of  destruction. 


Tools 

Smart  board  and  Battlescape  good. 

How  it  would  transition  to  reality  is  still  a  question. 

Improvements:  there  is  too  much  symbology  and  it  needs  to  be  easier  to  manipulate. 
Still  a  huge  improvement  on  pins  and  maps. 
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Gave  superb  SA  with  respect  to  movement  about  the  battlefield.  Mastering  the  symbology 
would  make  it  more  effective.  SA  has  definitely  sped  up  the  whole  process  (allowed  multi 
rolling  to  be  used) 

Analysis  Function  not  being  performed  in  Bde  HQ.  Data  would  be  turned  into  useful  info 
by  S2  and  that  sent  to  pilots.  Need  the  S2  role  to  be  more  real  so  analysed,  contextual  data 
sent. 

Able  to  do  more  with  SA  and  therefore  some  mission  objectives  failed  and  too  much  was 
done  without  the  plan.  S2  would  be  able  to  control  and  slow  down  info  flow  when 
necessary  and  advise  on  way  to  proceed. 

Pilots 

Tools 

■  In  reality  both  pilots  are  SA  aware 

■  Front  pilot  can't  give  info  to  back  pilot 

■  Limitation  of  current  system 

Information  Sharing  between  helicopters?  This  is  normal  through  radio  comms. 

There  is  no  capability  to  designate  en  and  'pass  off'.  If  it  could,  it  will  appear  on  everyone's 
screen  (if  using  a  data  link  would  be  realistic)  or  would  normally  be  sent  verbally. 

Pilot  broadcasts  when  he  decides  -  but  Bde  can  request  info. 

Continuos  update  in  position  not  realistic. 

BDE  HQ  doesn't  want  to  know  where  every  entity  is.  Just  Company  HQ  for  eg. 

Back  brief  from  Battle  map  on  Screen 
BDE  HQ 

■  Airmobile  went  particularly  well,  consisted  as  planned 

■  ARH  very  successful  in  bottom  LOCUST  part 

■  North  preliminary  operation  worked  well 

■  Tick  and  Flea  worked  well  and  able  to  use  2C  AV  -  which  was  unplanned  but  very 
effective 

■  CAIRS  not  available  and  PGM  artillery  also  not  available 

■  Distracted  ARH  from  NTH  to  clear  en  GBAD  NAI2  -well  supported  by  SF 

■  Caught  out  when  ARH  came  down  and  took  out  en  C2  very  quickly  and  were 
unprepared. 

■  Way  ahead  from  Manbuloo  not  secure  yet 

■  That's  when  the  corralling  of  en  started 

■  Locating  the  en  Tank  company  on  the  W  side  of  the  river  was  a  surprise  which 
required  redirected  effort  (eg)  ARH  effort 

■  Should  have  used  artillery  on  them 


49 


DSTO-TR-1521 


■  Speed  of  the  ARH  was  the  trigger  to  bring  forward  H  hr 

■  Effort  of  ARH's  focussed  on  C2  and  destruction  of  the  reserve. 

■  Fear  of  a  pause  in  the  battle  for  en  to  regroup  and  destroy  crossings  -  the  Bridge 
already  being  out  was  a  surprise  -  had  to  adjust  groups  MITRE  and  SABRE  so  that 
MITRE  had  to  go  around  and  cross  at  Knots  Crossing  instead. 

■  A  gridline  limit  was  imposed  on  MITRE  until  sure  that  Phase  1  was  complete 


JOSCC 

■  ARH  to  clear  area  -  expected  to  be  there  and  they  were 

■  Stopped  short  about  1  km  of  Manbuloo  and  had  two  simulated  battles  to  get  inside 
en  decision  cycle  and  ensure  he  fights  2  battles 

■  GBAD  and  C2  destruction  were  event  driven 

■  Concerned  that  intent  was  not  relaying  across  system  and  resulted  in  loss  of  2 
ARH's 

■  Used  pincer  movement  to  destroy  T80's 

■  Need  graphics  to  support  voice  comms 

■  Didn't  want  the  ARH's  to  engage  that  close 

■  Analysis  question  for  future:  can  you  pass  intent  down  a  radio  -  or  do  you  need 
graphics  (SA)  also? 

■  Pilots  are  often  given  eastings  and  northings  to  stay  clear  of  -  such  as  here.  That 
intent  can  be  passed  by  radio.  It  was  the  method  of  execution  that  changed  not  the 
meaning  of  the  intent. 

■  Are  these  methods  sufficient  given  SA  tools? 

■  SQN  HQ  can  designate  no  fly  area  -  can  depend  on  the  skill  of  the  pilot. 

■  Pilot  needs  discretion  to  accept  or  reject  a  digital  update  eg  re:  EW  threats.  As  soon 
as  a  restriction  update  (eg  no  fly  zone)  is  added,  then  it  can't  be  rejected  as  it's  a 
control  measure.  Receipt  has  to  be  acknowledged  (ACK) 

■  Machine  to  machine  ACK  is  not  sufficient  like  Human  to  Human  ACK. 

■  If  control  measure  is  in  then  it  needs  to  be  accepted. 

■  Pilot  must  ACK  that  he  understands 

■  Info  via  digital  ok,  but  need  the  verbal  comm  of  ACK  of  new  control  measure 

■  Helo's  internal  net  needs  button  to  represent  'ACK'  and  stop  verbal  'Roger  Sir' 

■  Was  interesting  to  see  what  the  helo's  would  do  once  they  crossed  the  no  fly  line 

*  Effectively,  some  human  needs  to  say  that  I  got  hat  and  that  I  understand  it. 

■  Confirmed  that  the  'ack'  button  needs  to  be  added. 

SQN  HQ(NORTH) 

■  All  conducted  in  accordance  with  RGT  COMD  orders,  was  modified  only  after  res 
destruction 

■  New  mission,  close  support  for  battle  Group  on  Line  of  Departure  and  find  and 
engage  T80's  E  of  Katherine 

■  Only  significant  change  was  the  quick  execution  of  GBAD  in  NAD 

Because  it  was  so  quick  and  effective,  was  any  thought  given  to  dwelling  until  airmobile 
ready?  No.  Was  a  deviation  in  the  plan.  Airmobile  didn't  impact  on  mission.  May  have 
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caused  synchronisation  issues  at  Bde  -  may  have  wanted  him  to  delay  -  S2  might  have 
provided  that  info. 

SQNHQ(SOUTH) 

■  Only  deviation  was  en  route  to  cut  off  res  force  needed  to  route  to  destroy  3  T80's. 

■  Then  flexed  right  and  targets  of  opportunity  and  continued  to  Katherine  and  lost  2 
Air. 

■  Did  the  SA  screen  show  targets  to  ARH's?  targets  didn't  come  up  ,  saw  no  en 
Aviation  will  not  always  have  a  reserve  and  Avn  COMD  adapted  accordingly. 

Trigger  for  launching  2  res  ARH's  -  T80's  -  high  value  targets 

Red  entities  always  appear  on  screen  in  Bde  HQ  even  when  dead  and  made  confusing. 
Planned  to  change  dead  entities  to  black. 

Helicopters 

•  Route  recon  and  airmobile 

■  Target  not  identified  by  UAV  so  general  sweep  performed 

■  Proceeded  according  to  plan 

■  2  res  ARH  to  move  up  and  act  as  screen  but  didn't  arrive,  so  stayed  in  position 
acting  as  screen 

■  Direction  to  engage  targets  of  opportunity  to  Katherine 

■  Took  a  'bull  in  a  china  shop'  approach  to  Katherine  which  led  to  death  -  unsure 
how  died,  didn't  see. 

■  Encountered  some  technical  difficulties 


Situation  Awareness 


ARHs 

Source  of  S A 

Passed  between  other  ARH  and  ID  of  en  on  screen  ie  SA  display.  Cross-referenced 
between  them. 

How  maintained  SA? 

Same  way 

How  was  the  data  fused? 

Friendly,  en  info  correlated  with  SA  to  what  Intel  on  TV  and  FOV. 

Was  this  achievable  or  was  the  info  too  much?  Fairly  realistic  for  flight  info. 

Sharing  information/SA? 

Unless  laser  designation  used,  then  by  conventional  radio  comms 

Did  you  share  with  any  other  units?  No.  Just  helo  and  HQ.  Not  aware  of  any  other 

friendly  units  in  vicinity. 

At  1150  H2  found  en  Manpad  and  engaged.  HI  flew  between  them.  HI  were  you  aware  of 
your  loc  and  H2  were  you  aware  of  His  locn? 

H2  told  HI  on  radio  -  but  too  busy  getting  rid  of  manpad  to  get  him  out  of  way. 
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HI  not  aware  of  his  loc. 

When  Helo  finds  target,  can't  make  it  appear  on  screen.  Have  to  lase  and  communicate 
manually  to  other  helo.  Need  a  menu  to  add  this  as  mentioned  before. 

Currently  can't  keep  track  of  all  targets  that  appear. 

Tp  COMD 

Source  ofSA 

Recon  source  SA.  Sensors  overlap.  Confirmed  route  clear.  Used  SA  tools,  especially  for 
other  air  loc  and  for  most  sensor  info. 

The  less  and  more  impt  info  that  can  be  displayed  on  screen  for  pilot  the  less  the  hazard. 
As  a  planning  tool  for  given  routes  the  area  covered  information  would  be  good,  but  not 
as  a  real  time  display. 

How  maintained  SA? 

Looking  at  moving  map  display  and  sensors 

When  in  Close  proximity  to  other  air,  do  you  intuitively  know  who's  who?  No,  have  to 
look  at  symbology. 

How  was  the  data  fused? 

TADS  system  a  priority  (targets  and  who's  trying  to  kill  me) 

Then  mission  info  then  radio 
Sharing  information/S A? 

Other  helo  -  voice,  map  display 

Didn't  do  this  with  designation  -  maybe  should  have  (b/ w  helo) 

Ergonomic  issue:  back  pilot  needs  to  twist  to  look  at  sensors  info  -  not  readily  available. 


SQNHQ 

Source  of  SA 

What's  on  map,  using  their  FOV  sensors,  radio  in  background  for  SITREPS 
No  direct  feed  from  UAV  -  would  have  liked. 

North  HQ  didn't  see  his  air  on  the  screen.  Using  LOCON  ModSAF  to  determine  what  was 
supposed  to  be  displayed. 

How  maintained  SA? 

Monitoring  source 
How  was  the  data  fused? 

Intuitively.  Understand  what  the  battle  plan  is  and  visualise  what  it  is.  Didn't  have  to  look 
at  screen  that  much  to  gain  SA. 

Rehearsal,  planning,  Synch  matrix,  bulk  of  tools  for  SA. 

Sharing  information/SA? 

Info  primarily  shared  with  Regt  COMD  with  Exception  of  SF  (but  that  was  mostly 
LOCON).  Verbally  over  radio. 

Even  if  Regt  COMD  has  moving  map,  still  needs  verbal  comms  ie  SITREPS  to  confirm  the 
picture  in  his  mind. 

Replay  -  very  quick  and  stopped  early.  Info  if  required  available. 
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ID  Targets  3  Tiers:  Detection,  recognition  and  identification 

Helos,  was  this  played  out  today?  Detection  of  en  in  LZ  initially  -  ?  of  en  and  friendly 
wearing  same  cam.  No  real  answer  here. 

Technical  and  Future  Concepts 

Any  extra  comments  can  be  added  in  books. 

HUD's  (  Head  up  Displays)  to  simulate  HUD  for  helos'  here?  A  good  idea?  Yes.  Also, 
contrast  colour  for  info  on  screen  eg  black  strip  on  bottom  with  white  text. 

Want  a  hot  mike  system  for  between  pilots  in  same  helo  and  a  footswitch  system  for 
comms  b/w  helos. 

Augmented  reality  concept  has  tremendous  application  and  will  be  available  for  next  expt. 
End  o  f  AAR  1. 


C.2.  AAR  2:  Mission  3  Wednesday  20/3/01 


Manoeuvre  COMD  brief 

No  real  change  to  plan.  Synchronised  with  battle  command.  Attacking  concentrated  in  the 
northern  area.  Launched  and  went  right  with  timing  through  to  SNAPPER  where 
GRASHOPPER  couldn't  designate  for  FA/ 18.  Were  able  to  get  it  going  though  and  GBAD 
was  successfully  destroyed.  C2  then  destroyed  inadvertently  and  through  'bull  in  china 
shop'  pushed  enemy  (en)  out  of  town.  Had  to  do  a  weapons  hold  to  control  a  little  and 
monitor  reserves  (res). 

More  coordination  with  fast  air  and  artillery  needed  -  but  is  coming  along  well. 

Used  impromptu  fire  lines  and  control  measures.  Were  effective  and  intent  was  realised 
with  hold  fire. 

Mission  was  quite  effective  and  was  achieved  today  as  opposed  to  yesterday. 

BDE  COMD  brief 

No  real  change  to  yesterday.  Synchronisation  issues  with  CAIRS  and  when 
GRASSHOPPER  lost  laser  designation. 

Significant  change  was  when  ARH  at  BASS,  chose  not  to  engage  res  when  found.  Was 
trying  to  avoid  the  corralling  of  yesterday  (opportunity  for  them  to  get  out).  Did  engage 
later. 

Speed  and  effectiveness  of  ARH  caused  changes  to  synchronisation  and  how  things  were 
worked. 
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SQN  COMD  (NORTH)  brief 

No  real  change,  primary  task  the  same.  Difference  was  not  to  actually  destroy  the  GBAD  - 
CAIRS.  Coordination  with  Grasshopper  was  significant  -  then  laser  designation  went 
down.  In  battle  position  and  prepared  to  engage,  but  necessarily  prepared  to  be  able  to 
designate  for  CAIRS.  Not  particularly  good  SA  at  Sqn  HQ  at  that  time,  worked  off  the 
synch.  Matrix.  Timings  were  early  -  no  SA  graphics  gave  indication. 

C2  nodes,  WEEVIL  had  effective  designation  and  ARH  Tp  removed. 

Not  receiving  SA  of  all  call  signs. 

Order  to  locate  and  destroy  reserve.  Some  confusion  with  pilots  when  told  TAM  when  it 
was  TAI2  and  TAI2  had  been  mixed  up  with  objective  4.  Sorted  out  by  SQN  CMD  and 
successful. 

Full  achievement  of  mission  not  completed  by  weapons  hold. 

Effectiveness  was  okay  and  there  were  gaps  in  the  SA. 

How  did  you  work  around  the  SA  loss?  Used  the  radio,  but  at  times,  bits  were  lots  with 
other  messages  cutting  over  the  top. 

Need  some  toll  indicating  what  targets  are  engaged  by  ARH  to  be  100%  sure. 

Is  it  a  danger  with  the  tools  to  micromanage  the  pilots?  Yes.  Easy  to  get  too  involved  -  but 
you're  not  the  man  on  the  ground. 

SA  of  environment  -  should  be  used  to  synchronise  at  the  command  levels  -  not 
micromanage. 

SQN  COMD  (SOUTH)  brief 

Involved  some  hasty  attack  operations  as  well  as  the  usual. 

Battlescape  needs  a  notional  reports  issue  eg  when  the  landing  Zone  (LZ)  was  clear  of 
threats. 

Synchronisation  coordination  was  via  the  Regt  as  was  flow  of  info.  Can't  do  spot  reports  in 
this  system  as  you  can  do  in  the  field. 

ARH  COMD  brief 

2  air  from  Tindal.  Confusing  as  to  which  aircraft  were  following  which  helo. 

Destroy  GBAD  at  TUNA  and  then  C2  at  Katherine.  Monitor  and  destroy  res  NW.  en 
route  to  1st  task  didn't  see  GBAD,  1st  target  was  armoured  (Armd)  vehicle.  'Bull  in  china 
shop'  issue  -  didn't  exactly  stick  to  plan.  Didn't  see  any  C2.  Armd  vehicles  and  res  only 
targets  seen.  Location  resulted  in  SA  breakdown  ie  SA  screen  showed  further  Sth  than 
actual  location.  Didn't  relate  laser  point  to  where  were  on  map. 

Synchronisation  issues  b/ w  helos,  gnd  tps,  HQ?  Communications  between  and  inter  helo 
were  good.  No  problems. 

Issue:  targets  grid  form  -  map  didn't  have  northing /eastings  easily  seen  and  sot  he  screen 
had  to  be  manipulated  which  made  things  difficult. 

WEEVIL  informed  Bde  HQ  that  the  C2  was  destroyed,  but  he  helo's  didn't  realise  they  had 
been  taken  out. 
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Sqn  HQ  didn't  think  they  would  see  the  nodes.  WEEVIL  had  eyes  on  the  target  and 
designated  for  it  and  were  the  confirmation.  Sqn  HQ  aware  of  C2  taken  out. 

Mission  was  effective. 

ACE  brief 

1st  problem  was  the  lack  of  a  link  to  AOD.  Therefore  jets  modelled  in  ModSAF.  TAOC 
passed  to  ACE  who  passed  to  FAC  for  strike.  Pilots  seemed  happy.  Directive  had  been 
given  earlier.  Ground  based  laser  designation  issue  (GRASSHOPPER).  Held  off.  Then 
rectified  and  they  came  in  and  successfully  destroyed  GBAD.  FAC  passed  back  to  ACE 
who  then  passed  back  to  TAOC. 

Artillery  (arty)  coordination  and  grids  and  terminology  -  had  some  issues,  but  an  SME 
assisted. 

SA  Tools  used?  None  really.  Used  SA  from  radio  and  those  around. 

Not  happy  with  the  JOSCC  SA  tools  provided  to  know  arty,  guns,  firing,  tracks,  airmobile 
etc  for  coordination  and  control.  Eg  don't  want  to  send  airmobile  through  an  engagement 
area.  Would  allow  things  to  happen  quicker  and  with  more  confidence. 

LOCON  brief 

GRASSHOPPER  laser  designation  issue  in  NAI2.  Still  don't  know  what  caused  it. 

BDA's  were  received  okay. 

WEEVIL  designation  effective.  Resulted  in  ~50%hit  success  with  laser  designation  and 
further  hits  to  target. 

WEEVIL  contacted  and  so  had  to  break  designation  on  C2  nodes. 

LOCUST  successfully  took  out  Manbuloo. 

FALCON  was  effective  in  taking  out  T80's  with  ARH  support. 

UAV  brief 

Focussed  on  N  instead  of  S  this  time.  Got  to  locations  eventually. 

Clash  with  airspace  issue  with  air  force 

Had  to  then  go  too  low  to  clear  out  and  got  shot  down. 

2nd  UAV  -  issues  with  controls  and  unrealistic  feel  (as  per  yesterdays  AAR) 

was  able  to  ID  some  targets  and  confirm  the  high  level  bridge  was  out -but  that  came  late. 

Useful  tool  with  a  mixed  bag  of  success.  Hasn't  had  to  search  for  targets.  Would  use 
standard  helo  tactics  for  search. 

En  brief 
No  comment. 
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FA/18  Bombing  Activity 

Capt.  Thomas  with  input  form  others.  Coupled  with  a  battle  map. 

At  H-80  left  to  move  to  SNAPPER,  until  H-60  when  CAIRS  supposed  to  join.  CAIRS  had 
been  called  in  earlier  and  he  was  not  aware  of  this.  They  then  thought  they  had  to 
designate  for  CAIRS  when  ground  based  designation  didn't  work.  Wouldn't  have  made  it 
in  time  if  the  designation  wasn't  fixed.  Trip  to  SNAPPER  was  a  lot  quicker  than  expected. 
Able  to  get  everything  to  work  in  the  end. 

What  would  have  been  needed  if  the  ARH's  did  have  to  do  the  laser  designation?  SF  grids 
and  it  would  have  taken  longer  to  acquire  the  targets.  Location  info  would  have  assisted  in 
the  grid  points.  Time  to  detect  the  anti-air  asset  would  have  delayed.  Moving  into  the 
threat  envelope  would  have  increased  the  risk  to  the  asset.  Laser  code  for  designation 
would  have  been  transferred. 

Any  use  of  threat  domes?  No.  But  the  information  there  is  still  invaluable  and  is  planned 
to  be  used  later.  Simpler  if  ARH  using  weapons  to  attack  to  hold  off  and  use  craft  over 
max  height. 


SA  TOOLS 

Going  around  the  room  to  collect  any  additional  information. 

Problem:  ModSAF  operator  killing  targets,  but  they  reappear  and  so  lost  confidence  in  the 
system.  This  has  been  fixed. 

Problem:  clearing  display  from  tracks,  then  cleared  areas  of  interest  and  therefore  lost  SA 
and  couldn't  get  them  back  -  too  busy  to  work  it  out. 

When  SA  lost  used  comms,  higher  command  and  remembering. 

More  control  measures  need  to  be  in  place  (introduced  from  planning  phase)  for  when  SA 
is  lost. 

Conflict  over  thoughts  for  reality  now  and  reality  10  yrs  from  now. 

SA  is  still  a  great  and  tremendous  improvement.  Looked  at  concepts  of  battle  tracking  and 
attrition  tables.  As  en  killed  be  able  to  cross  off  to  know  how  many  of  theirs  are  left  etc.  for 
decision  making.  Gives  a  battle  history  as  you  go.  Used  OPORD  overlays. 

Still  can't  do  grid  refs.  Have  more  faith  in  the  SA  tool  screen  than  on  the  old  map  boards 
and  pins  and  radio  transmission  and  someone  can  always  write  something  down  wrong 
and  transfer  that  info  thru  the  chain. 

Possibility  of  using  control  graphics  in  real  time-no  opportunity  today  as  would  take  too 
long.  At  the  moment  it's  quicker  to  do  by  radio.  Should  be  able  to  construct  graphic  on 
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screen  and  upload  to  ARH.  Mouse  or  smart  board  would  make  this  easy  with  out  dialogue 
boxes  etc.  Has  to  be  quick. 

Reading  Grids  issue.  If  using  GPS  be  prepared  for  confusion.  Confirmed  that  lat/long 
cords  can  be  converted  to  grids. 

Cockpit  workloads  were  at  a  max  when  chatter  was  at  a  max.  This  was  also  when  needed 
to  focus  on  SA  display  (look  for  grids)  and  therefore  lost  some  capability. 

Need  better  tools,  to  cut  out  some  radio  chatter  and  presentation  of  info  to  be  clear  and 
intuitive.  Eg  own  symbol  centred  always  on  screen. 

Wrong  grid  reference  came  out  of  this  -  400m  difference  -  enough  to  kill.  (118035  instead 
of  118039) 

Confusion  with  SA  display  showing  wrong  location  of  helos  to  pilots. 

3rd  party  designation.  Did  you  have  any  knowledge  that  missile  would  be  in  footprint?  No. 
Programmed  in  missile.  Just  point  and  shoot.  No  control. 


Points  from  MONDAY  1913101 

Bde  HQ  orders  given  using  smartboard  but  besides  that  wasn't  much  different  to  a  WWII 
HQ.  Could  be  better  with  the  tools  available. 

Eg  program  in  plan  and  watch  it  unfold  as  an  orders  tool  and  adjust  accordingly.  Ie 
wargame  and  order  at  same  time  -  allows  rehearsal  and  adjustment. 

Currently  underway,  should  be  ready  for  next  expt. 

LTCOL  Chris  Bums  did  not  participate  in  WWII. 

Agreed  that  there  is  so  much  scope  and  that  only  about  10%  of  capability  was  used. 

Good  to  see  from  last  year  that  things  are  developing  in  a  simple  way  and  'back  to  basics'. 
Simplicity  is  the  key:  need  an  area  a  job  and  the  resources  to  do  it. 

Need  to  get  away  from  cluttered  screens  -  too  complicated  to  clear  -  only  simple 
techniques.  The  aggregation  of  icons  would  be  good. 

EW  Discussion 

Sensors  were  used  for  en  info  -  how  can  you  be  sure  that  en  hasn't  interrogated  sensors  so 
you  see  info?  How  do  you  know  if  a  ref  (grid)  of  en  target  is  true?  Eg  creating  a  false 
entity? 

Air  used  a  lot  of  visual  ID  -  but  if  relying  on  sensors  then  that's  another  story. 
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Reliant  on  correct  designation  and  en  not  doing  it  to  us.  (with  codes  etc  chances  are  very 
low) 

No  confirmation  in  answer  to  question. 

For  Gnd  Comd  unless  Intel  has  sensor  picking  up  emission  then  usually  forces  make 
contact  and  fight. 

Need  to  get  phantoms  into  system  for  experience  and  training  and  allow  practice  with 
reprioritising  and  chaos. 

Allow  a  process  for  confidence  and  validity  to  be  introduced.  Outside  scope  of  task  but 
still  beneficial. 

EW  part  of  the  bigger  package. 

Idea  of  working  and  using  EW  is  to  be  prepared  and  trained  for  it  so  when  confronted 
with  it  can  deal  with  it. 

DSTO  Comments 

EW  issues  for  bigger  exptl  picture  -  outside  scope  of  task,  but  can  value  add  -  need 
stakeholder  acceptance. 

SA  display  comments  have  been  thought  of  and  are  under  development  -  trying  to 
incorporate  for  next  expt. 

Asked  military  to  think  about  what  issues  might  be  learned  about  training?  Initial 
responses:  practice  with  target  designation  by  ground  and  more  visual  flying  in  sim. 

Were  the  systems  reasonable  intuitive?  Sim  was  fine,  screens  ok.  Just  takes  some  adjusting 
in  what  is  seen  and  correlating  to  mission.  Would  be  good  to  fly  some  training  sorties  on 
the  battlefield  for  orientation. 

Would  a  facility  for  training  similar  to  this  be  good?  Yes.  UK  are  currently  using  a  sim  for 
Comanche  training  successfully. 

Comment:  pilots  flew  good  mission  today  in  SERF  allowing  for  short  time  using  them 
Impact  of  Tools  and  Land  Air  Systems  on  CAIRS: 

Changes  in  procedure  resulted  from  SA. 

Jets  were  held  at  contact  while  laser  restart  could  redirect  designation. 

Could  SA  speed  up  traditional  current  CAIRS  mission?  Yes,  if  you  have  a  backup  plan. 
Need  lots  of  flexibility  for  evolving  tactical  plans.  CAIRS  missions  need  5  layers  of 
redundancy,  so  need  flexible  systems  and  tools. 
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When  laser  was  down,  gave  small  insight  into  the  changes  needed  to  flexible  plans.  Good 
FAC  would  have  contingencies  already  planned. 

Mission  4  will  have  ARH  close  to  C2.  In  real  ops,  whether  ARH  in  close  or  jets  used 
depends  on  the  weapon  required.  Only  use  CAIRS  if  need  that  weapon. 

Effect  of  weapon  system/platform  on  battlefield  used  to  achieve  an  effect  -  not  just 
because  it's  there. 

End  of  AAR  2. 


C.3.  AAR  3:  Mission  4  Thursday  22/03/01 

ARH 

What  was  the  mission? 

■  Route  for  the  helicopters 

■  Primary  task  -  clear  route  to  Manbuloo 

■  Security  for  force  troops  on  the  ground 
End  state 

Landing  Zone  (LZ)  at  Manbuloo  secured  to  continue  operations 
Tactics 

■  Both  to  clear  route  N/NE 

■  Performed  normal  techniques  within  limits  of  sensors  and  visual  range 

Weapons/Sensors 

■  Combination  of  all  sensors 

■  Air  to  ground  missiles  (3000m  range) 

■  Cannon  for  ranges  less  than  this 

Effectiveness 

■  Covered  the  lack  of  reality 

■  Unrealistic  weapons  choice  (eg  hellfire  used  on  manpads) 

■  Not  as  effective  against  single  enemy  targets 

SQN  HQ  (NORTH) 

What  was  the  mission? 

■  Destroy  the  enemy  (en)  C2  node/ s 

*  Preliminary  operation:  clear  NAI2  (GBAD)  in  conjunction  with  SF 

■  Phase  1:  destroy  C2  at  TAI  (West  of  Katherine) 

■  Locate  and  destroy  en  reserve(res) 
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End  state 

GBAD,  NAI2  destroyed /neutralised 
En  C2  nodes/HQ  destroyed 
Res  destroyed 

Tactics 

■  Maximum  standoff  range 

■  Ground  based  laser  designation  for  1st  two  targets 

■  Res  combination,  recon  and  standoff  range 

Weapons/Sensors 

■  Mainly  hellfire  and  ground  based  designation  (SF) 
Effectiveness 

Achieved  end  states  and  was  effective. 

SQNHQ  (SOUTH) 

What  was  the  mission? 

■  Air  assault  security 

■  Secure  assault  elements 

■  Security  for  ground  forces 

Weapons/Sensors 

■  Hellfire  and  cannon 

■  UAV's  (couldn't  tell  where  the  target  was  from  it) 

Effectiveness 

■  Fair 


UAVs 

What  was  the  mission? 

Locate  en  in  Manbuloo  and  the  res  force 

Tasks 

■  Locate  en  in  Manbuloo 

■  Prepare  to  lase  targets  for  ARH 

■  Find  res 

End  state 

Secure  airmobile  operation  and  identify  res 
Tactics 

■  Descend  through  to  area  move/ascend  to  ensure  out  of  weapons  range  then 
continue  to  next  position 

■  Had  to  unrealistically  descend  to  get  good  resolution 
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■  Were  able  to  fly  by  Manbuloo  and  to  reserve 

■  If  still  able,  would  have  stayed  with  airmobile  area  until  retasked 

■  Lack  of  control  of  UAV.  Is  there  a  technical  glitch  here? 

■  Actual  plan  didn't  happen 

BDEHQ 

What  was  the  mission? 

■  Capture  Katherine  and  dislodge  en,  destroy  defence  and  en  reserve  to  N  of 
Katherine 

Task 

■  Establish  strong  SF  and  Armd  recon  screen 

■  Secure  Manbuloo 

■  Destroy  GBAD 

*  Destroy  en  C2  and  res 

■  Attack  Katherine  -  dislodge  en  -attack  NW  Kath  and  destroy  en. 

End  state 

Successfully  capture  Katherine  with  minimum  damage  to  infrastructure.  Destroy/ capture 
en  NW  of  Katherine. 

Tactics 

■  Establish  strong  SF/Armd  screen  early 

■  Conduct  preliminary  operation  -airmobile  to  Manbuloo 

■  Destruction  of  GBAD 

■  Destruction  of  C2/  res 

■  2  phase  attack 

■  phase  1-  Katherine 

■  phase  2  -  area  NW  of  Katherine 

Weapons/Sensors 

■  SF,  ARH,  Helos,  Armd  Recon,  UAV,  Mechanised  and  Motorised  Infantry,  guns, 
artillery 

Effectiveness 

■  Very  effective  battle  group 

■  Very  effective  ARH,  SF  and  Armd  Recon 

■  UAV  not  as  effective  as  hoped 

■  Control  of  UAV  -  JOSCC 

■  Was  almost  too  effective-  didn't  rout  en  in  Katherine  -instead  corralled  them  there. 

■  Perhaps  should  have  been  less  aggressive  to  west  of  river. 

■  Current  scenario  would  not  have  minimised  the  impact  of  destruction. 


Tools 

Smart  board  and  Battlescape  good. 

How  it  would  transition  to  reality  is  still  a  question. 
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Improvements:  there  is  too  much  symbology  and  it  needs  to  be  easier  to  manipulate. 
Still  a  huge  improvement  on  pins  and  maps. 

End  of  AAR  3 
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